Alternatives and similar repositories for triton-grpc-proxy-rs:

Users that are interested in triton-grpc-proxy-rs are comparing it to the libraries listed below

• recmo / cria
  Tiny inference-only implementation of LLaMA
  ☆92Updated 10 months ago
• ialacol / text-inference-batcher
  A high performance batching router optimises max throughput for text inference workload
  ☆16Updated last year
• FL33TW00D / embd
  GPU accelerated client-side embeddings for vector search, RAG etc.
  ☆65Updated last year
• deployradiant / pychatml
  Chat Markup Language conversation library
  ☆55Updated last year
• taylorai / onnx_embedding_models
  utilities for loading and running text embeddings with onnx
  ☆44Updated 6 months ago
• apehex / tokun
  Tokun to can tokens
  ☆16Updated this week
• sdan / selfextend
  an implementation of Self-Extend, to expand the context window via grouped attention
  ☆118Updated last year
• lightblue-tech / lb-reranker
  ☆20Updated 3 weeks ago
• cognitivecomputations / kraken
  ☆65Updated 8 months ago
• mixedbread-ai / batched
  The Batched API provides a flexible and efficient way to process multiple requests in a batch, with a primary focus on dynamic batching o…
  ☆122Updated 2 months ago
• enjalot / fineweb-modal
  Using modal.com to process FineWeb-edu data
  ☆20Updated 2 months ago
• thooton / muse
  Let's create synthetic textbooks together :)
  ☆73Updated last year
• Not-Diamond / RoRF
  Routing on Random Forest (RoRF)
  ☆114Updated 4 months ago
• LLukas22 / llm-rs-python
  Unofficial python bindings for the rust llm library. 🐍❤️🦀
  ☆75Updated last year
• michaelfeil / embed
  A stable, fast and easy-to-use inference library with a focus on a sync-to-async API
  ☆45Updated 4 months ago
• ashvardanian / usearch-binary
  Binary vector search example using Unum's USearch engine and pre-computed Wikipedia embeddings from Co:here and MixedBread
  ☆18Updated 10 months ago
• charlesfrye / minimodal
  A miniature version of Modal
  ☆19Updated 8 months ago
• simonw / llm-embed-jina
  Embedding models from Jina AI
  ☆58Updated last year
• premAI-io / benchmarks
  🕹️ Performance Comparison of MLOps Engines, Frameworks, and Languages on Mainstream AI Models.
  ☆136Updated 6 months ago
• teknium1 / transformers-gptq-quant
  ☆48Updated last year
• nisten / grokadamw
  new optimizer
  ☆19Updated 6 months ago
• raphaelsty / LeNLP
  NLP with Rust for Python 🦀🐍
  ☆61Updated 8 months ago
• Hellisotherpeople / llm_steer-oobabooga
  Steer LLM outputs towards a certain topic/subject and enhance response capabilities using activation engineering by adding steering vecto…
  ☆43Updated 11 months ago
• abetlen / program-constrained-language-model-sampling
  ☆34Updated last year
• JD-P / RetroInstruct
  Synthetic data derived by templating, few shot prompting, transformations on public domain corpora, and monte carlo tree search.
  ☆30Updated 2 months ago
• davanstrien / haiku-dpo
  Using open source LLMs to build synthetic datasets for direct preference optimization
  ☆57Updated 11 months ago
• teknium1 / LLM-Logbook
  Public reports detailing responses to sets of prompts by Large Language Models.
  ☆29Updated last month
• vikhyat / mixtral-inference
  inference code for mixtral-8x7b-32kseqlen
  ☆99Updated last year