Alternatives and similar repositories for vLLM-Benchmark

Users that are interested in vLLM-Benchmark are comparing it to the libraries listed below

• LMCache / lmcache-vllm
  The driver for LMCache core to run in vLLM
  ☆41Updated 4 months ago
• Michaelvll / llm-ie-benchmarks
  A collection of reproducible inference engine benchmarks
  ☆31Updated last month
• bytedance-iaas / splitwise-demos
  ☆25Updated 3 months ago
• foundation-model-stack / fastsafetensors
  High-performance safetensors model loader
  ☆36Updated this week
• run-ai / llmperf
  ☆54Updated 8 months ago
• InternLM / turbomind
  ☆85Updated 2 months ago
• WukLab / preble
  Stateful LLM Serving
  ☆70Updated 2 months ago
• project-etalon / etalon
  LLM Serving Performance Evaluation Harness
  ☆78Updated 3 months ago
• bytedance / InfiniStore
  KV cache store for distributed LLM inference
  ☆254Updated last week
• AlibabaPAI / torchacc
  PyTorch distributed training acceleration framework
  ☆49Updated 3 months ago
• Azure / msccl-executor-nccl
  ☆37Updated 5 months ago
• ademeure / DeeperGEMM
  DeeperGEMM: crazy optimized version
  ☆69Updated last month
• abcdabcd987 / libfabric-efa-demo
  ☆37Updated 5 months ago
• fw-ai / benchmark
  Benchmark suite for LLMs from Fireworks.ai
  ☆75Updated 3 weeks ago
• tensorchord / deepseek-api-arena
  A benchmarking tool for comparing different LLM API providers' DeepSeek model deployments.
  ☆29Updated 2 months ago
• HPMLL / NVIDIA-Hopper-Benchmark
  ☆41Updated last week
• cchan / tccl
  extensible collectives library in triton
  ☆87Updated 2 months ago
• anyscale / llm-continuous-batching-benchmarks
  ☆119Updated last year
• infinigence / Semi-PD
  A prefill & decode disaggregated LLM serving framework with shared GPU memory and fine-grained compute isolation.
  ☆81Updated 2 weeks ago
• danyangz / lightning
  Lightning In-Memory Object Store
  ☆46Updated 3 years ago
• octoml / octoml-profile
  Home for OctoML PyTorch Profiler
  ☆113Updated 2 years ago
• mlc-ai / mlc-python
  ☆34Updated last week
• bentoml / llm-bench
  ☆52Updated 6 months ago
• microsoft / ParrotServe
  [OSDI'24] Serving LLM-based Applications Efficiently with Semantic Variable
  ☆158Updated 8 months ago
• Azure / msccl
  Microsoft Collective Communication Library
  ☆65Updated 6 months ago
• vllm-project / flash-attention
  Fast and memory-efficient exact attention
  ☆72Updated last month
• feifeibear / LLMRoofline
  Compare different hardware platforms via the Roofline Model for LLM inference tasks.
  ☆100Updated last year
• ppl-ai / pplx-kernels
  Perplexity GPU Kernels
  ☆324Updated 2 weeks ago
• cisco-open / pymultiworld
  A framework for PyTorch to enable fault management for collective communication libraries (CCL) such as NCCL
  ☆19Updated last month
• Infini-AI-Lab / MagicDec
  [ICLR2025] Breaking Throughput-Latency Trade-off for Long Sequences with Speculative Decoding
  ☆116Updated 6 months ago