Alternatives and similar repositories for vLLM-Benchmark

Users that are interested in vLLM-Benchmark are comparing it to the libraries listed below

• Michaelvll / llm-ie-benchmarks
  A collection of reproducible inference engine benchmarks
  ☆31Updated 2 months ago
• LMCache / lmcache-vllm
  The driver for LMCache core to run in vLLM
  ☆42Updated 4 months ago
• bytedance-iaas / splitwise-demos
  ☆26Updated 3 months ago
• ademeure / DeeperGEMM
  DeeperGEMM: crazy optimized version
  ☆69Updated last month
• tensorchord / inference-benchmark
  Benchmark for machine learning model online serving (LLM, embedding, Stable-Diffusion, Whisper)
  ☆28Updated last year
• tensorchord / deepseek-api-arena
  A benchmarking tool for comparing different LLM API providers' DeepSeek model deployments.
  ☆28Updated 2 months ago
• InternLM / turbomind
  ☆86Updated 3 months ago
• foundation-model-stack / fastsafetensors
  High-performance safetensors model loader
  ☆40Updated this week
• HPMLL / NVIDIA-Hopper-Benchmark
  ☆44Updated 3 weeks ago
• abcdabcd987 / libfabric-efa-demo
  ☆39Updated 5 months ago
• mlc-ai / mlc-python
  ☆35Updated last month
• infinigence / Semi-PD
  A prefill & decode disaggregated LLM serving framework with shared GPU memory and fine-grained compute isolation.
  ☆87Updated last month
• Azure / msccl-executor-nccl
  ☆37Updated 6 months ago
• chips-compilers-mlsys-21 / chips-compilers-mlsys-21.github.io
  ☆11Updated 4 years ago
• cchan / tccl
  extensible collectives library in triton
  ☆86Updated 2 months ago
• bytedance / InfiniStore
  KV cache store for distributed LLM inference
  ☆269Updated 2 weeks ago
• bentoml / llm-bench
  ☆54Updated 7 months ago
• run-ai / llmperf
  ☆55Updated 9 months ago
• vllm-project / flash-attention
  Fast and memory-efficient exact attention
  ☆76Updated this week
• WukLab / preble
  Stateful LLM Serving
  ☆73Updated 3 months ago
• deepspeedai / DeepSpeed-Kernels
  ☆72Updated 3 months ago
• Azure / msccl
  Microsoft Collective Communication Library
  ☆64Updated 7 months ago
• cisco-open / pymultiworld
  A framework for PyTorch to enable fault management for collective communication libraries (CCL) such as NCCL
  ☆19Updated last month
• octoml / octoml-profile
  Home for OctoML PyTorch Profiler
  ☆113Updated 2 years ago
• flashinfer-ai / cutlass-viz
  ☆60Updated 2 months ago
• ryantd / veloce
  WIP. Veloce is a low-code Ray-based parallelization library that makes machine learning computation novel, efficient, and heterogeneous.
  ☆18Updated 2 years ago
• Infini-AI-Lab / MagicDec
  [ICLR2025] Breaking Throughput-Latency Trade-off for Long Sequences with Speculative Decoding
  ☆116Updated 6 months ago
• danyangz / lightning
  Lightning In-Memory Object Store
  ☆46Updated 3 years ago
• hpcaitech / TensorNVMe
  A Python library transfers PyTorch tensors between CPU and NVMe
  ☆116Updated 7 months ago
• coreweave / ml-containers
  ☆37Updated this week