simon-mo / vLLM-BenchmarkLinks
☆31Updated 9 months ago
Alternatives and similar repositories for vLLM-Benchmark
Users that are interested in vLLM-Benchmark are comparing it to the libraries listed below
Sorting:
- Genai-bench is a powerful benchmark tool designed for comprehensive token-level performance evaluation of large language model (LLM) serv…☆263Updated this week
- The driver for LMCache core to run in vLLM☆60Updated last year
- Toolchain built around the Megatron-LM for Distributed Training☆84Updated 2 months ago
- ☆96Updated 10 months ago
- Home for OctoML PyTorch Profiler☆113Updated 2 years ago
- ☆56Updated last year
- Accepted to MLSys 2026☆70Updated last week
- LLM Serving Performance Evaluation Harness☆83Updated 11 months ago
- torchcomms: a modern PyTorch communications API☆327Updated this week
- A TUI-based utility for real-time monitoring of InfiniBand traffic and performance metrics on the local node☆63Updated last month
- Pretrain, finetune and serve LLMs on Intel platforms with Ray☆131Updated 4 months ago
- Fast and memory-efficient exact attention☆111Updated last week
- A NCCL extension library, designed to efficiently offload GPU memory allocated by the NCCL communication library.☆90Updated last month
- A collection of reproducible inference engine benchmarks☆38Updated 9 months ago
- ☆71Updated 10 months ago
- High-performance safetensors model loader☆99Updated 3 weeks ago
- DeepXTrace is a lightweight tool for precisely diagnosing slow ranks in DeepEP-based environments.☆92Updated 3 weeks ago
- DeeperGEMM: crazy optimized version☆73Updated 9 months ago
- A prefill & decode disaggregated LLM serving framework with shared GPU memory and fine-grained compute isolation.☆123Updated last month
- ☆77Updated last year
- ☆61Updated last year
- KV cache store for distributed LLM inference☆390Updated 2 months ago
- Benchmark suite for LLMs from Fireworks.ai☆89Updated last week
- Offline optimization of your disaggregated Dynamo graph☆177Updated last week
- NVIDIA NVSHMEM is a parallel programming interface for NVIDIA GPUs based on OpenSHMEM. NVSHMEM can significantly reduce multi-process com…☆462Updated last month
- A unified library for building, evaluating, and storing speculative decoding algorithms for LLM inference in vLLM☆220Updated this week
- A high-performance RL training-inference weight synchronization framework, designed to enable second-level parameter updates from trainin…☆131Updated last month
- ☆125Updated last year
- ☆47Updated last year
- extensible collectives library in triton☆95Updated 10 months ago