High-performance Rust benchmark client for vLLM serving endpoints.
☆43Jul 9, 2026Updated this week
Alternatives and similar repositories for vllm-bench
Users that are interested in vllm-bench are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Early-stage Rust drop-in alternative frontend for vLLM☆71May 22, 2026Updated last month
- [Accepted to SOSP 2026] Fast Deterministic LLM Inference☆21Updated this week
- An LLM post-training framework with vLLM for RL Scaling☆363Updated this week
- ☆44Updated this week
- MLIR-based TileLang Ascend Adapter☆21Updated this week
- Managed Database hosting by DigitalOcean • AdPostgreSQL, MySQL, MongoDB, Kafka, Valkey, and OpenSearch available. Automatically scale up storage and focus on building your apps.
- Pure Rust + CUDA LLM inference engine — no PyTorch, OpenAI-compatible, serves Qwen3 to Kimi-K2☆536Updated this week
- GPU Functional Descriptor for memory access☆34May 24, 2026Updated last month
- libsmctrl论文的复现，添加了python端接口，可以在python端灵活调用接口来分配计算资源☆12May 21, 2024Updated 2 years ago
- High performance RMSNorm Implement by using SM Core Storage(Registers and Shared Memory)☆30Jan 22, 2026Updated 5 months ago
- DeepXTrace is a lightweight tool for precisely diagnosing slow ranks in DeepEP-based environments.☆100Jan 16, 2026Updated 5 months ago
- Batched square compact-Householder QR factorization.☆15Jul 2, 2026Updated last week
- [MLSys 26] 🥇 Solution for Gated Delta Net Track of MLSys 26 Flash infer competition☆35May 22, 2026Updated last month
- A high-performance and light-weight router for vLLM large scale deployment☆309Updated this week
- Genai-bench is a powerful benchmark tool designed for comprehensive token-level performance evaluation of large language model (LLM) serv…☆311Jun 24, 2026Updated 2 weeks ago
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- ☆55Apr 30, 2025Updated last year
- vLLM plugin for block-based diffusion language model (dLLM) support☆23May 25, 2026Updated last month
- Common recipes to run vLLM☆914Updated this week
- ☆20May 30, 2026Updated last month
- ☆32Jul 2, 2025Updated last year
- helm charts for deploying models with llm-d☆31Jun 27, 2026Updated 2 weeks ago
- SQL Benchmark derived from TPC-H☆11May 20, 2023Updated 3 years ago
- Document Automation Reference Kit☆16Jun 27, 2024Updated 2 years ago
- TokenSpeed is a speed-of-light LLM inference engine.☆1,589Updated this week
- Serverless GPU API endpoints on Runpod - Get Bonus Credits • AdSkip the infrastructure headaches. Auto-scaling, pay-as-you-go, no-ops approach lets you focus on innovating your application.
- ☆40Jul 1, 2026Updated last week
- ☆148Aug 18, 2025Updated 10 months ago
- Debug print operator for cudagraph debugging☆18Aug 2, 2024Updated last year
- Community maintained hardware plugin for vLLM on Spyre☆52Updated this week
- The vLLM XPU kernels for Intel GPU☆53Updated this week
- ☆21Mar 3, 2025Updated last year
- A TUI-based utility for real-time monitoring of InfiniBand traffic and performance metrics on the local node☆71May 16, 2026Updated last month
- Experimental JIT compiler generator☆29Apr 17, 2024Updated 2 years ago
- A framework and CLI toolkit for orchestrating teams of loosely-coupled AI agents.☆17Updated this week
- Proton VPN Special Offer - Get 70% off • AdSpecial partner offer. Trusted by over 100 million users worldwide. Tested, Approved and Recommended by Experts.
- Compare different hardware platforms via the Roofline Model for LLM inference tasks.☆123Mar 13, 2024Updated 2 years ago
- An efficient GPU support for LLM inference with x-bit quantization (e.g. FP6,FP5).☆282Jul 16, 2025Updated 11 months ago
- mKernel: fast multi-node, multi-GPU fused kernels☆251Jun 21, 2026Updated 3 weeks ago
- ☆17Apr 10, 2026Updated 3 months ago
- Agnostic is a Rust ecosystem that provides runtime-agnostic abstractions for async operations. Write your async code once and run it seam…☆26Updated this week
- NVIDIA Inference Benchmarks provide recipes in ready-to-use templates for evaluating platform speed. Validate your platform across speci…☆37Updated this week
- High-performance KV cache storage for LLM inference — GPU offloading, SSD caching, and cross-node sharing via RDMA. Works with vLLM and S…☆172Updated this week