Alternatives and similar repositories for vLLM-Benchmark

Users that are interested in vLLM-Benchmark are comparing it to the libraries listed below

• LMCache / lmcache-vllm
  The driver for LMCache core to run in vLLM
  ☆40Updated 3 months ago
• Michaelvll / llm-ie-benchmarks
  A collection of reproducible inference engine benchmarks
  ☆30Updated 3 weeks ago
• ademeure / DeeperGEMM
  DeeperGEMM: crazy optimized version
  ☆69Updated last week
• InternLM / turbomind
  ☆84Updated last month
• cchan / tccl
  extensible collectives library in triton
  ☆86Updated last month
• bentoml / llm-bench
  ☆50Updated 5 months ago
• fw-ai / benchmark
  Benchmark suite for LLMs from Fireworks.ai
  ☆73Updated this week
• mlc-ai / mlc-python
  ☆32Updated this week
• Infini-AI-Lab / MagicDec
  [ICLR2025] Breaking Throughput-Latency Trade-off for Long Sequences with Speculative Decoding
  ☆116Updated 5 months ago
• foundation-model-stack / fastsafetensors
  High-performance safetensors model loader
  ☆30Updated last month
• microsoft / ParrotServe
  [OSDI'24] Serving LLM-based Applications Efficiently with Semantic Variable
  ☆155Updated 7 months ago
• abcdabcd987 / libfabric-efa-demo
  ☆34Updated 4 months ago
• WukLab / preble
  Stateful LLM Serving
  ☆67Updated 2 months ago
• Azure / msccl-executor-nccl
  ☆36Updated 5 months ago
• deepspeedai / DeepSpeed-Kernels
  ☆69Updated last month
• MDK8888 / vllmini
  A minimal implementation of vllm.
  ☆40Updated 9 months ago
• IBM / triton-dejavu
  Framework to reduce autotune overhead to zero for well known deployments.
  ☆70Updated this week
• IST-DASLab / Sparse-Marlin
  Boosting 4-bit inference kernels with 2:4 Sparsity
  ☆73Updated 8 months ago
• vedantroy / gpu_kernels
  ☆26Updated last year
• project-etalon / etalon
  LLM Serving Performance Evaluation Harness
  ☆78Updated 2 months ago
• chips-compilers-mlsys-21 / chips-compilers-mlsys-21.github.io
  ☆11Updated 4 years ago
• microsoft / AttentionEngine
  ☆70Updated last week
• infinigence / Semi-PD
  A prefill & decode disaggregated LLM serving framework with shared GPU memory and fine-grained compute isolation.
  ☆73Updated 2 weeks ago
• vllm-project / flash-attention
  Fast and memory-efficient exact attention
  ☆68Updated 2 weeks ago
• ScalingIntelligence / hydragen
  Hydragen: High-Throughput LLM Inference with Shared Prefixes
  ☆36Updated last year
• anyscale / llm-continuous-batching-benchmarks
  ☆117Updated last year
• microsoft / TileFusion
  TileFusion is an experimental C++ macro kernel template library that elevates the abstraction level in CUDA C for tile processing.
  ☆85Updated this week
• flashinfer-ai / cutlass-viz
  ☆58Updated 3 weeks ago
• pytorch / tlparse
  TORCH_LOGS parser for PT2
  ☆37Updated 3 weeks ago
• danyangz / lightning
  Lightning In-Memory Object Store
  ☆45Updated 3 years ago