Alternatives and similar repositories for triton-grpc-proxy-rs

Users that are interested in triton-grpc-proxy-rs are comparing it to the libraries listed below

• ialacol / text-inference-batcher
  A high performance batching router optimises max throughput for text inference workload
  ☆16Updated last year
• LLukas22 / llm-rs-python
  Unofficial python bindings for the rust llm library. 🐍❤️🦀
  ☆75Updated last year
• taylorai / onnx_embedding_models
  utilities for loading and running text embeddings with onnx
  ☆44Updated 10 months ago
• charlesfrye / minimodal
  A miniature version of Modal
  ☆20Updated last year
• Narsil / bloomserver
  ☆39Updated 2 years ago
• DeployQL / LintDB
  Vector Database with support for late interaction and token level embeddings.
  ☆55Updated 8 months ago
• tiiuae / onebitllms
  Lightweight toolkit package to train and fine-tune 1.58bit Language models
  ☆78Updated last month
• runpod-workers / worker-sglang
  SGLang is fast serving framework for large language models and vision language models.
  ☆23Updated 4 months ago
• teknium1 / LLM-Logbook
  Public reports detailing responses to sets of prompts by Large Language Models.
  ☆30Updated 5 months ago
• interstellarninja / function-calling-eval
  A framework for evaluating function calls made by LLMs
  ☆37Updated 11 months ago
• cognitivecomputations / kraken
  ☆66Updated last year
• michaelfeil / embed
  A stable, fast and easy-to-use inference library with a focus on a sync-to-async API
  ☆45Updated 8 months ago
• euclaise / SlimTrainer
  Full finetuning of large language models without large memory requirements
  ☆94Updated last year
• enjalot / latent-data-modal
  Using modal.com to process FineWeb-edu data
  ☆20Updated 2 months ago
• recmo / cria
  Tiny inference-only implementation of LLaMA
  ☆93Updated last year
• mixedbread-ai / batched
  The Batched API provides a flexible and efficient way to process multiple requests in a batch, with a primary focus on dynamic batching o…
  ☆137Updated last month
• facebookresearch / fastgen
  Simple high-throughput inference library
  ☆119Updated last month
• premAI-io / benchmarks
  🕹️ Performance Comparison of MLOps Engines, Frameworks, and Languages on Mainstream AI Models.
  ☆137Updated 10 months ago
• abetlen / program-constrained-language-model-sampling
  ☆35Updated 2 years ago
• leo-du / llama2.rs
  Inference Llama 2 in one file of zero-dependency, zero-unsafe Rust
  ☆38Updated last year
• deployradiant / pychatml
  Chat Markup Language conversation library
  ☆55Updated last year
• firstbatchxyz / function-calling-eval
  The DPAB-α Benchmark
  ☆25Updated 5 months ago
• catena-labs / moa-llm
  A Python library to orchestrate LLMs in a neural network-inspired structure
  ☆49Updated 8 months ago
• guidance-ai / llgtrt
  TensorRT-LLM server with Structured Outputs (JSON) built with Rust
  ☆55Updated last month
• sdan / selfextend
  an implementation of Self-Extend, to expand the context window via grouped attention
  ☆119Updated last year
• IntrinsicLabsAI / grammar-builder
  Generates grammer files from typescript for LLM generation
  ☆38Updated last year
• thesephist / spectre
  Sparse autoencoders for Contra text embedding models
  ☆25Updated last year
• furiousteabag / vram-calculator
  Transformer GPU VRAM estimator
  ☆65Updated last year
• thooton / muse
  Let's create synthetic textbooks together :)
  ☆75Updated last year
• NousResearch / StripedHyenaTrainer
  ☆61Updated last year