Early-stage Rust drop-in alternative frontend for vLLM
☆71May 22, 2026Updated last month
Alternatives and similar repositories for vllm-frontend-rs
Users that are interested in vllm-frontend-rs are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- High-performance Rust benchmark client for vLLM serving endpoints.☆43Updated this week
- High-performance KV cache storage for LLM inference — GPU offloading, SSD caching, and cross-node sharing via RDMA. Works with vLLM and S…☆172Updated this week
- An LLM post-training framework with vLLM for RL Scaling☆363Updated this week
- Pure Rust + CUDA LLM inference engine — no PyTorch, OpenAI-compatible, serves Qwen3 to Kimi-K2☆536Updated this week
- Engine-agnostic LLM gateway in Rust. Full OpenAI & Anthropic API compatibility across vLLM, TRT-LLM, TokenSpeed, SGLang, OpenAI, Gemini &…☆391Updated this week
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- A Triton JIT runtime and ffi provider in C++☆37Updated this week
- A high-performance and light-weight router for vLLM large scale deployment☆309Updated this week
- mKernel: fast multi-node, multi-GPU fused kernels☆251Jun 21, 2026Updated 3 weeks ago
- SQL Benchmark derived from TPC-H☆11May 20, 2023Updated 3 years ago
- NVIDIA Inference Xfer Library (NIXL)☆1,133Updated this week
- High Performance KV Cache Store for LLM☆58May 20, 2026Updated last month
- TokenSpeed is a speed-of-light LLM inference engine.☆1,589Updated this week
- Terminal UI for NVIDIA Nsight Systems profiles — timeline viewer, kernel navigator, NVTX hierarchy☆64Updated this week
- AIPerf is a comprehensive benchmarking tool that measures the performance of generative AI models served by your preferred inference solu…☆441Updated this week
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- 15-721 Spring 2024 - Cache #1☆12May 2, 2024Updated 2 years ago
- Tools for generating TPC-* datasets☆32Jun 23, 2024Updated 2 years ago
- A userspace filesystem backing by Apache OpenDAL.☆39Jun 2, 2026Updated last month
- Memory-bounded compressed sparse attention via streaming top-k. Triton kernels for the DeepSeek-V4 lightning indexer. 32x regime extensio…☆22May 5, 2026Updated 2 months ago
- ☆13Jan 7, 2025Updated last year
- ☆44Updated this week
- Example of applying CUDA graphs to LLaMA-v2☆11Aug 25, 2023Updated 2 years ago
- Simple python library for generating your own perfetto traces for your application. Can be used for both app instrumentation and custom …☆26Jun 22, 2025Updated last year
- ML kernels and benchmarking infrastructure written in TIRx☆65Updated this week
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- 通过实验对比LLM推理中Prefill和Decoding阶段的吞吐量差异，揭示性能瓶颈，解释PD分离优化技术的原理。包含CUDA和Apple MPS (M系列芯片) 的测试脚本。☆22May 22, 2025Updated last year
- Testbed for LLM inference with cutile-rs.☆65Jul 1, 2026Updated last week
- Deriving steepest descent convergence bounds and hyperparameter scaling laws in machine learning optimization from first principles, form…☆16Apr 11, 2026Updated 3 months ago
- CMU-DB's Cascades optimizer framework☆405Jan 6, 2025Updated last year
- SGLang Omni: High-Performance Multi-Stage Pipeline Framework for Omni Models☆627Updated this week
- vLLM plugin for block-based diffusion language model (dLLM) support☆23May 25, 2026Updated last month
- ☆153Updated this week
- ☆67Updated this week
- The vLLM XPU kernels for Intel GPU☆53Updated this week
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- This CLI tool and Python3 module collects the current system state for documentation☆27Apr 9, 2026Updated 3 months ago
- A User-Defined Function Framework for Apache Arrow.☆113Apr 22, 2026Updated 2 months ago
- A lightweight design for computation-communication overlap.☆242Jan 20, 2026Updated 5 months ago
- Skills for writing tilelang and debugging with CUDA toolkits.☆130May 20, 2026Updated last month
- ☆303Jun 9, 2026Updated last month
- ✨A list of awesome TiDB patterns, code templates, demos✨☆20Jun 27, 2022Updated 4 years ago
- Perplexity GPU Kernels☆588Nov 7, 2025Updated 8 months ago