Early-stage Rust drop-in alternative frontend for vLLM
☆73May 22, 2026Updated 5 months ago
Alternatives and similar repositories for vllm-frontend-rs
Users that are interested in vllm-frontend-rs are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- High-performance Rust benchmark client for vLLM serving endpoints.☆53Aug 3, 2026Updated 3 months ago
- BlitzScale Router - Distributed LLM Inference Router (Rust)☆18May 25, 2026Updated 5 months ago
- High-performance KV cache storage for LLM inference — GPU offloading, SSD caching, and cross-node sharing via RDMA. Works with vLLM and S…☆221Updated this week
- vLLM Daily Summarization of Merged PRs☆56Updated this week
- A high-performance and light-weight router for vLLM large scale deployment☆511Updated this week
- GPU virtual machines on DigitalOcean Gradient AI • AdGet to production fast with high-performance AMD and NVIDIA GPUs you can spin up in seconds. The definition of operational simplicity.
- vLLM Model plugin for the encoder-decoder BART model☆13Updated this week
- Pure Rust + CUDA LLM inference engine — no PyTorch, OpenAI-compatible, serves Qwen3 to Kimi-K2☆739Updated this week
- A Triton JIT runtime and ffi provider in C++☆41Nov 4, 2026Updated last week
- An LLM post-training framework with vLLM for RL Scaling☆503Updated this week
- mKernel: fast multi-node, multi-GPU fused kernels☆309Oct 17, 2026Updated 3 weeks ago
- Apache OpenDAL™ YinYang: a cross-platform filesystem foundation.☆54Nov 3, 2026Updated last week
- SQL Benchmark derived from TPC-H☆11May 20, 2023Updated 3 years ago
- htop-like TUI for real-time RDMA network monitoring.☆129Updated this week
- NVIDIA Inference Xfer Library (NIXL)☆1,363Updated this week
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- A collection of megakernels for TPU☆185Oct 15, 2026Updated 3 weeks ago
- Engine-agnostic LLM gateway in Rust. Full OpenAI & Anthropic API compatibility across vLLM, TRT-LLM, TokenSpeed, SGLang, OpenAI, Gemini &…☆712Updated this week
- Common recipes to run vLLM☆1,085Updated this week
- `oli` stands for OpenDAL Command Line Interface. It aims to provide a unified and user-friendly way to manipulate data stored in various …☆27Oct 3, 2026Updated last month
- ☆28Mar 31, 2026Updated 7 months ago
- ☆29Updated this week
- the frontend of lmcache☆17Apr 22, 2026Updated 6 months ago
- TokenSpeed is a speed-of-light LLM inference engine.☆2,918Updated this week
- A PyTorch native library for training speculative decoding models☆267Updated this week
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- 15-721 Spring 2024 - Cache #1☆12May 2, 2024Updated 2 years ago
- Fast and memory-efficient exact attention☆23Updated this week
- ☆143Updated this week
- ☆15Mar 6, 2026Updated 8 months ago
- Tools for generating TPC-* datasets☆33Jun 23, 2024Updated 2 years ago
- Composable and Embeddable Communication Runtime for Distributed AI Services☆103Oct 7, 2026Updated last month
- Repository for go shared libraries (for now).☆11Nov 3, 2026Updated last week
- ☆15Jul 10, 2026Updated 4 months ago
- ☆19Aug 19, 2024Updated 2 years ago
- End-to-end encrypted cloud storage - Proton Drive • AdSpecial offer: 40% Off Yearly / 80% Off First Month. Protect your most important files, photos, and documents from prying eyes.
- GLM-5.3-Flash adaptation + offload perf patches for FreeToken v0.1.2 (17.8→36.7 tok/s on one RTX PRO 6000)☆21Oct 2, 2026Updated last month
- Performance benchmark & accuracy evaluation for vLLM☆18Nov 4, 2026Updated last week
- PegaFlow is a high-performance KV cache offloading solution for vLLM v1 on single-node multi-GPU setups.☆25Jan 7, 2026Updated 10 months ago
- High Performance KV Cache Store for LLM☆59May 20, 2026Updated 5 months ago
- ☆97Sep 15, 2025Updated last year
- Bitwise-identical token probabilities between RL rollout (vLLM) and training☆40Updated this week
- DeepSelect: TopK kernels for DeepSeek Sparse Attention (DSA) and Samplers☆540Sep 30, 2026Updated last month
Are these results useful?