Early-stage Rust drop-in alternative frontend for vLLM
☆73May 22, 2026Updated 5 months ago
Alternatives and similar repositories for vllm-frontend-rs
Users that are interested in vllm-frontend-rs are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- High-performance Rust benchmark client for vLLM serving endpoints.☆52Aug 3, 2026Updated 2 months ago
- High-performance KV cache storage for LLM inference — GPU offloading, SSD caching, and cross-node sharing via RDMA. Works with vLLM and S…☆217Updated this week
- A high-performance and light-weight router for vLLM large scale deployment☆480Updated this week
- vLLM Daily Summarization of Merged PRs☆56Oct 12, 2026Updated last week
- ☆26Updated this week
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- Pure Rust + CUDA LLM inference engine — no PyTorch, OpenAI-compatible, serves Qwen3 to Kimi-K2☆724Updated this week
- A Triton JIT runtime and ffi provider in C++☆41Sep 24, 2026Updated 3 weeks ago
- An LLM post-training framework with vLLM for RL Scaling☆490Oct 14, 2026Updated last week
- Engine-agnostic LLM gateway in Rust. Full OpenAI & Anthropic API compatibility across vLLM, TRT-LLM, TokenSpeed, SGLang, OpenAI, Gemini &…☆661Updated this week
- Apache OpenDAL™ YinYang: a cross-platform filesystem foundation.☆52Sep 25, 2026Updated 3 weeks ago
- mKernel: fast multi-node, multi-GPU fused kernels☆296Updated this week
- htop-like TUI for real-time RDMA network monitoring.☆122Updated this week
- NVIDIA Inference Xfer Library (NIXL)☆1,325Updated this week
- SQL Benchmark derived from TPC-H☆11May 20, 2023Updated 3 years ago
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- ☆58Apr 22, 2026Updated 6 months ago
- the frontend of lmcache☆17Apr 22, 2026Updated 6 months ago
- A model-agnostic GPU runtime for shipping models as verified programs.☆132Updated this week
- `oli` stands for OpenDAL Command Line Interface. It aims to provide a unified and user-friendly way to manipulate data stored in various …☆27Oct 3, 2026Updated 2 weeks ago
- Performance benchmark & accuracy evaluation for vLLM☆17Oct 15, 2026Updated last week
- ☆28Mar 31, 2026Updated 6 months ago
- TokenSpeed is a speed-of-light LLM inference engine.☆2,754Updated this week
- Model Express is a Rust-based component meant to be placed next to existing model inference systems to speed up their startup times and i…☆192Updated this week
- ☆81Oct 15, 2026Updated last week
- 1-Click AI Models by DigitalOcean Gradient • AdDeploy popular AI models on DigitalOcean Gradient GPU virtual machines with just a single click. Zero configuration with optimized deployments.
- A PyTorch native library for training speculative decoding models☆266Updated this week
- Common recipes to run vLLM☆1,066Updated this week
- ☆19Oct 8, 2026Updated 2 weeks ago
- ☆116Aug 5, 2026Updated 2 months ago
- Tools for generating TPC-* datasets☆33Jun 23, 2024Updated 2 years ago
- 15-721 Spring 2024 - Cache #1☆12May 2, 2024Updated 2 years ago
- Fast and memory-efficient exact attention☆23Jun 26, 2026Updated 3 months ago
- Composable and Embeddable Communication Runtime for Distributed AI Services☆101Oct 7, 2026Updated 2 weeks ago
- ☆97Sep 15, 2025Updated last year
- Serverless GPU API endpoints on Runpod - Get Bonus Credits • AdSkip the infrastructure headaches. Auto-scaling, pay-as-you-go, no-ops approach lets you focus on innovating your application.
- Memory-bounded compressed sparse attention via streaming top-k. Triton kernels for the DeepSeek-V4 lightning indexer. 32x regime extensio…☆27May 5, 2026Updated 5 months ago
- PegaFlow is a high-performance KV cache offloading solution for vLLM v1 on single-node multi-GPU setups.☆25Jan 7, 2026Updated 9 months ago
- High Performance KV Cache Store for LLM☆58May 20, 2026Updated 5 months ago
- A high-performance, RDMA-native data-plane library for large-scale heterogeneous memory pools, developed by the Tencent Astral Network Te…☆33Aug 6, 2026Updated 2 months ago
- A workload for deploying LLM inference services on Kubernetes☆309Updated this week
- Example of applying CUDA graphs to LLaMA-v2☆11Aug 25, 2023Updated 3 years ago
- UCCL is an efficient communication library for GPUs, covering collectives, P2P (e.g., KV cache transfer, RL weight transfer), and EP (e.g…☆1,550Updated this week