A high-performance and light-weight router for vLLM large scale deployment
☆188Apr 12, 2026Updated this week
Alternatives and similar repositories for router
Users that are interested in router are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- High performance RMSNorm Implement by using SM Core Storage(Registers and Shared Memory)☆30Jan 22, 2026Updated 2 months ago
- ☆15Jul 9, 2024Updated last year
- Cute layout visualization☆32Jan 18, 2026Updated 2 months ago
- 15-721 Spring 2024 - Cache #1☆12May 2, 2024Updated last year
- NVIDIA Inference Xfer Library (NIXL)☆970Updated this week
- Deploy open-source AI quickly and easily - Bonus Offer • AdRunpod Hub is built for open source. One-click deployment and autoscaling endpoints without provisioning your own infrastructure.
- Model Express is a Rust-based component meant to be placed next to existing model inference systems to speed up their startup times and i…☆52Updated this week
- ☆28Apr 17, 2025Updated 11 months ago
- KV cache store for distributed LLM inference☆405Nov 13, 2025Updated 5 months ago
- A lightweight, configurable, and real-time simulator designed to mimic the behavior of vLLM without the need for GPUs or running actual h…☆113Apr 9, 2026Updated last week
- ☆28Mar 17, 2024Updated 2 years ago
- A better wrapper for using RDMA programming APIs in Rust flavor☆80Updated this week
- MultiArchKernelBench: A Multi-Platform Benchmark for Kernel Generation☆52Mar 25, 2026Updated 3 weeks ago
- Kubernetes enhancements for Network Topology Aware Gang Scheduling & Autoscaling☆194Updated this week
- ☆12Mar 18, 2024Updated 2 years ago
- Serverless GPU API endpoints on Runpod - Bonus Credits • AdSkip the infrastructure headaches. Auto-scaling, pay-as-you-go, no-ops approach lets you focus on innovating your application.
- Efficient, Flexible, and Highly Fault-Tolerant Model Service Management Based on SGLang☆61Nov 8, 2024Updated last year
- ☆20Jun 9, 2025Updated 10 months ago
- ☆15Feb 23, 2025Updated last year
- ☆98May 31, 2025Updated 10 months ago
- A unified library for building, evaluating, and storing speculative decoding algorithms for LLM inference in vLLM☆335Updated this week
- Expert Specialization MoE Solution based on CUTLASS☆26Jan 19, 2026Updated 2 months ago
- llm-d helm charts and deployment examples☆54Apr 2, 2026Updated 2 weeks ago
- We apply attention on LSTM to give more weight to the items which are more relevant for next-item prediction☆11Dec 26, 2018Updated 7 years ago
- Efficient and easy multi-instance LLM serving☆543Mar 12, 2026Updated last month
- Deploy open-source AI quickly and easily - Bonus Offer • AdRunpod Hub is built for open source. One-click deployment and autoscaling endpoints without provisioning your own infrastructure.
- Building LaTeX packages using Travis-CI☆12Dec 21, 2019Updated 6 years ago
- Lock-free buddy allocator based on binary heap☆13Mar 3, 2025Updated last year
- Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.☆5,071Updated this week
- Keyaki Treebank Parsed Corpus☆10May 15, 2019Updated 6 years ago
- Kubernetes CSI Driver for serving OCI model artifacts☆25Mar 23, 2026Updated 3 weeks ago
- FlexAttention w/ FlashAttention3 Support☆27Oct 5, 2024Updated last year
- ☆15Jan 24, 2022Updated 4 years ago
- Supercharge Your LLM with the Fastest KV Cache Layer☆7,969Updated this week
- Perplexity GPU Kernels☆565Nov 7, 2025Updated 5 months ago
- Wordpress hosting with auto-scaling - Free Trial • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- A userspace filesystem backing by Apache OpenDAL.☆36Jan 8, 2026Updated 3 months ago
- A parallelism VAE avoids OOM for high resolution image generation☆90Mar 12, 2026Updated last month
- ☆18Mar 25, 2026Updated 3 weeks ago
- NVSHMEM‑Tutorial: Build a DeepEP‑like GPU Buffer☆177Feb 11, 2026Updated 2 months ago
- [NSDI '24] DINT: Fast In-Kernel Distributed Transactions with eBPF☆53Jul 6, 2024Updated last year
- buck2 oci rules☆11Apr 8, 2026Updated last week
- ☆10Apr 20, 2025Updated 11 months ago