Alternatives and similar repositories for triton-grpc-proxy-rs

Users that are interested in triton-grpc-proxy-rs are comparing it to the libraries listed below

• ialacol / text-inference-batcher
  A high performance batching router optimises max throughput for text inference workload
  ☆16Updated 2 years ago
• taylorai / onnx_embedding_models
  utilities for loading and running text embeddings with onnx
  ☆44Updated 4 months ago
• LLukas22 / llm-rs-python
  Unofficial python bindings for the rust llm library. 🐍❤️🦀
  ☆76Updated 2 years ago
• deployradiant / pychatml
  Chat Markup Language conversation library
  ☆55Updated last year
• recmo / cria
  Tiny inference-only implementation of LLaMA
  ☆92Updated last year
• teknium1 / transformers-gptq-quant
  ☆45Updated 2 years ago
• Preemo-Inc / text-generation-inference
  ☆198Updated last year
• euclaise / SlimTrainer
  Full finetuning of large language models without large memory requirements
  ☆94Updated 3 months ago
• notarussianteenager / srf-attention
  Simplex Random Feature attention, in PyTorch
  ☆75Updated 2 years ago
• sdan / selfextend
  an implementation of Self-Extend, to expand the context window via grouped attention
  ☆119Updated last year
• michaelfeil / embed
  A stable, fast and easy-to-use inference library with a focus on a sync-to-async API
  ☆46Updated last year
• vikhyat / mixtral-inference
  inference code for mixtral-8x7b-32kseqlen
  ☆104Updated 2 years ago
• abetlen / program-constrained-language-model-sampling
  ☆35Updated 2 years ago
• NolanoOrg / InstructLLaMa.cpp
  Fast inference of Instruct tuned LLaMa on your personal devices.
  ☆23Updated 2 years ago
• FL33TW00D / embd
  GPU accelerated client-side embeddings for vector search, RAG etc.
  ☆65Updated 2 years ago
• JD-P / RetroInstruct
  Synthetic data derived by templating, few shot prompting, transformations on public domain corpora, and monte carlo tree search.
  ☆32Updated 2 months ago
• DeployQL / LintDB
  Vector Database with support for late interaction and token level embeddings.
  ☆54Updated 6 months ago
• enjalot / latent-data-modal
  Using modal.com to process FineWeb-edu data
  ☆20Updated 8 months ago
• premAI-io / benchmarks
  🕹️ Performance Comparison of MLOps Engines, Frameworks, and Languages on Mainstream AI Models.
  ☆139Updated last year
• teknium1 / LLM-Benchmark-Logs
  Just a bunch of benchmark logs for different LLMs
  ☆119Updated last year
• Narsil / bloomserver
  ☆39Updated 3 years ago
• thooton / muse
  Let's create synthetic textbooks together :)
  ☆75Updated last year
• oKatanaaa / kolibrify
  Curriculum training of instruction-following LLMs with Unsloth
  ☆14Updated last week
• ashvardanian / JaccardIndex
  Optimizing bit-level Jaccard Index and Population Counts for large-scale quantized Vector Search via Harley-Seal CSA and Lookup Tables
  ☆21Updated 7 months ago
• furiousteabag / vram-calculator
  Transformer GPU VRAM estimator
  ☆67Updated last year
• official-elinas / zeus-llm-trainer
  Zeus LLM Trainer is a rewrite of Stanford Alpaca aiming to be the trainer for all Large Language Models
  ☆70Updated 2 years ago
• lightblue-tech / lb-reranker
  ☆24Updated 10 months ago
• mgerstgrasser / tacheles
  a lightweight, open-source blueprint for building powerful and scalable LLM chat applications
  ☆28Updated last year
• nicholasyager / llama-cpp-guidance
  A guidance compatibility layer for llama-cpp-python
  ☆36Updated 2 years ago
• adarshxs / TokenTally
  Estimate Your LLM's Token Toll Across Various Platforms and Configurations
  ☆38Updated last month