simon-mo / vLLM-BenchmarkLinks
☆31Updated 7 months ago
Alternatives and similar repositories for vLLM-Benchmark
Users that are interested in vLLM-Benchmark are comparing it to the libraries listed below
Sorting:
- Genai-bench is a powerful benchmark tool designed for comprehensive token-level performance evaluation of large language model (LLM) serv…☆247Updated this week
- The driver for LMCache core to run in vLLM☆59Updated 10 months ago
- ☆56Updated last year
- ☆97Updated 8 months ago
- Toolchain built around the Megatron-LM for Distributed Training☆79Updated last week
- A unified library for building, evaluating, and storing speculative decoding algorithms for LLM inference in vLLM☆149Updated this week
- torchcomms: a modern PyTorch communications API☆302Updated this week
- LLM Serving Performance Evaluation Harness☆82Updated 9 months ago
- Fast and memory-efficient exact attention☆104Updated this week
- Offline optimization of your disaggregated Dynamo graph☆121Updated this week
- Home for OctoML PyTorch Profiler☆114Updated 2 years ago
- DeepXTrace is a lightweight tool for precisely diagnosing slow ranks in DeepEP-based environments.☆73Updated this week
- Benchmark suite for LLMs from Fireworks.ai☆84Updated 3 weeks ago
- ☆58Updated last year
- ☆71Updated 8 months ago
- KV cache store for distributed LLM inference☆372Updated last month
- Efficient Compute-Communication Overlap for Distributed LLM Inference☆66Updated last month
- [NeurIPS 2025] Scaling Speculative Decoding with Lookahead Reasoning☆56Updated last month
- DeeperGEMM: crazy optimized version☆73Updated 7 months ago
- NVIDIA NVSHMEM is a parallel programming interface for NVIDIA GPUs based on OpenSHMEM. NVSHMEM can significantly reduce multi-process com…☆407Updated last month
- A prefill & decode disaggregated LLM serving framework with shared GPU memory and fine-grained compute isolation.☆118Updated 6 months ago
- ☆73Updated 11 months ago
- A collection of reproducible inference engine benchmarks☆38Updated 7 months ago
- ☆114Updated 6 months ago
- ☆122Updated last year
- OME is a Kubernetes operator for enterprise-grade management and serving of Large Language Models (LLMs)☆334Updated this week
- Ship correct and fast LLM kernels to PyTorch☆126Updated this week
- Pretrain, finetune and serve LLMs on Intel platforms with Ray☆130Updated 2 months ago
- ☆205Updated 7 months ago
- High-performance safetensors model loader☆79Updated 3 weeks ago