Alternatives and similar repositories for vllm-router:

Users that are interested in vllm-router are comparing it to the libraries listed below

• LLM360 / crystalcoder-data-prep
  Data preparation code for CrystalCoder 7B LLM
  ☆44Updated 8 months ago
• foundation-model-stack / bamba
  Train, tune, and infer Bamba model
  ☆80Updated 2 weeks ago
• Infini-AI-Lab / MagicDec
  [ICLR2025] Breaking Throughput-Latency Trade-off for Long Sequences with Speculative Decoding
  ☆105Updated last month
• FreedomIntelligence / FastLLM
  Fast LLM Training CodeBase With dynamic strategy choosing [Deepspeed+Megatron+FlashAttention+CudaFusionKernel+Compiler];
  ☆36Updated last year
• hetailang / SqueezeAttention
  ☆37Updated 3 months ago
• SkyworkAI / Skywork-MoE
  Skywork-MoE: A Deep Dive into Training Techniques for Mixture-of-Experts Language Models
  ☆128Updated 7 months ago
• nyunAI / PruneGPT
  ☆52Updated 7 months ago
• GreenBitAI / green-bit-llm
  A toolkit for fine-tuning, inferencing, and evaluating GreenBitAI's LLMs.
  ☆80Updated last week
• vllm-project / dashboard
  vLLM performance dashboard
  ☆20Updated 9 months ago
• samchaineau / llm_slerp_generation
  Repo hosting codes and materials related to speeding LLMs' inference using token merging.
  ☆34Updated 9 months ago
• 66RING / CritiPrefill
  Code repo for "CritiPrefill: A Segment-wise Criticality-based Approach for Prefilling Acceleration in LLMs".
  ☆13Updated 4 months ago
• 18907305772 / FuseAI
  FuseAI Project
  ☆80Updated this week
• wejoncy / QLLM
  A general 2-8 bits quantization toolbox with GPTQ/AWQ/HQQ/VPTQ, and export to onnx/onnx-runtime easily.
  ☆155Updated last week
• zhaochenyang20 / ModelServer
  Efficient, Flexible, and Highly Fault-Tolerant Model Service Management Based on SGLang
  ☆31Updated 2 months ago
• bentoml / BentoLMDeploy
  Self-host LLMs with LMDeploy and BentoML
  ☆17Updated last month
• astramind-ai / Mixture-of-depths
  Unofficial implementation for the paper "Mixture-of-Depths: Dynamically allocating compute in transformer-based language models"
  ☆145Updated 7 months ago
• wuhy68 / Parameter-Efficient-MoE
  Parameter-Efficient Sparsity Crafting From Dense to Mixture-of-Experts for Instruction Tuning on General Tasks
  ☆139Updated 4 months ago
• fw-ai / benchmark
  Benchmark suite for LLMs from Fireworks.ai
  ☆64Updated last month
• UbiquitousLearning / SLM_Survey
  ☆80Updated 3 months ago
• neuralmagic / AutoFP8
  ☆169Updated 3 months ago
• mistralai / mistral-evals
  ☆57Updated this week
• IST-DASLab / SparseFinetuning
  Repository for Sparse Finetuning of LLMs via modified version of the MosaicML llmfoundry
  ☆40Updated last year
• rayleizhu / vllm-ra
  [ACL 2024] RelayAttention for Efficient Large Language Model Serving with Long System Prompts
  ☆38Updated 11 months ago
• feifeibear / Odysseus-Transformer
  Odysseus: Playground of LLM Sequence Parallelism
  ☆64Updated 7 months ago
• InternLM / turbomind
  ☆59Updated last month
• golololologol / LLM-Distillery
  A pipeline for LLM knowledge distillation
  ☆85Updated this week
• jeffreysijuntan / lloco
  The official repo for "LLoCo: Learning Long Contexts Offline"
  ☆114Updated 7 months ago
• project-etalon / etalon
  LLM Serving Performance Evaluation Harness
  ☆66Updated 5 months ago
• pprp / Pruner-Zero
  Pruner-Zero: Evolving Symbolic Pruning Metric from scratch for LLMs
  ☆76Updated 2 months ago
• RobertCsordas / moe_attention
  Official repository for the paper "SwitchHead: Accelerating Transformers with Mixture-of-Experts Attention"
  ☆96Updated 4 months ago