vLLM Router
☆59Mar 11, 2024Updated 2 years ago
Alternatives and similar repositories for vllm-router
Users that are interested in vllm-router are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- All-in-one benchmarking platform for evaluating LLM.☆16Nov 12, 2025Updated 11 months ago
- Multiple GEMM operators are constructed with cutlass to support LLM inference.☆20Aug 3, 2025Updated last year
- Whisper in TensorRT-LLM☆16Sep 21, 2023Updated 3 years ago
- 中英文关键词抽取☆12Oct 28, 2024Updated 2 years ago
- NCU-driven iterative optimization workflow for CUDA/CUTLASS/Triton/CuTe DSL kernels.☆25Apr 10, 2026Updated 6 months ago
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- FA4-based Relative Attention Kernel developed by TML and Colfax☆18Sep 11, 2026Updated last month
- MegEngine implementation of Diffusion Models.☆19Aug 8, 2022Updated 4 years ago
- ☆16Aug 26, 2026Updated 2 months ago
- ☆11Oct 28, 2024Updated 2 years ago
- ☆65Apr 26, 2025Updated last year
- ☆89Apr 18, 2025Updated last year
- fake CUTLASS to get peformance☆26Apr 28, 2026Updated 6 months ago
- Experiments on Multi-Head Latent Attention☆102Aug 19, 2024Updated 2 years ago
- USTC-SGY 人工智能通识课 主页☆23Jun 19, 2025Updated last year
- End-to-end encrypted cloud storage - Proton Drive • AdSpecial offer: 40% Off Yearly / 80% Off First Month. Protect your most important files, photos, and documents from prying eyes.
- A lightweight design for computation-communication overlap.☆252Jan 20, 2026Updated 9 months ago
- ☆24Oct 10, 2025Updated last year
- Github repo for ICLR-2025 paper, Fine-tuning Large Language Models with Sparse Matrices☆26Feb 2, 2026Updated 9 months ago
- Analyze LLM inference: FLOPs, memory, Roofline model. Supports GQA, MoE, MLA, RoPE, SwiGLU. 19 models × 20+ hardware platforms.☆23Updated this week
- Expert Specialization MoE Solution based on CUTLASS☆27Apr 14, 2026Updated 6 months ago
- This repository contains the results and code for the MLPerf™ Training v3.0 benchmark.☆12Aug 10, 2023Updated 3 years ago
- A high-throughput and memory-efficient inference and serving engine for LLMs☆14Updated this week
- ☆136Nov 11, 2024Updated last year
- ☆32Jul 2, 2025Updated last year
- AI Agents on DigitalOcean Gradient AI Platform • AdBuild production-ready AI agents using customizable tools or access multiple LLMs through a single endpoint. Create custom knowledge bases or connect external data.
- ☆20Aug 27, 2026Updated 2 months ago
- FlagOS community governance, developer contribution guide, contribution agreement, and community communication | 包含FlagOS社区治理、开发者贡献指南、开发者…☆19Updated this week
- Benchmark tests supporting the TiledCUDA library.☆19Nov 19, 2024Updated last year
- High-performance GEMM implementation optimized for NVIDIA H100 GPUs, leveraging Hopper architecture's TMA, WGMMA, and Thread Block Cluste…☆11Aug 25, 2026Updated 2 months ago
- ☆21Updated this week
- A prefill & decode disaggregated LLM serving framework with shared GPU memory and fine-grained compute isolation.☆141Dec 25, 2025Updated 10 months ago
- Evaluate state-of-the-art sparse embedding models on the LIMIT dataset (`limit-small` and `limit`) from google's paper `On the Theoretica…☆16Sep 4, 2025Updated last year
- Kinematic and dynamic models of continuum and articulated soft robots.☆16Aug 13, 2026Updated 2 months ago
- An experimental communicating attention kernel based on DeepEP.☆34Jul 29, 2025Updated last year
- End-to-end encrypted cloud storage - Proton Drive • AdSpecial offer: 40% Off Yearly / 80% Off First Month. Protect your most important files, photos, and documents from prying eyes.
- Awesome Resources about MegEngine☆16Mar 2, 2023Updated 3 years ago
- ☆140Apr 23, 2024Updated 2 years ago
- 一个基于 Paddle Inference 封装的用于快速部署的高层 API☆33Nov 13, 2021Updated 4 years ago
- ☆17Apr 16, 2026Updated 6 months ago
- ☆12May 11, 2025Updated last year
- Agent-native Seedance 2.0 short-film studio: cli for AI, canvas for human☆17Sep 8, 2026Updated last month
- ☆14Dec 22, 2024Updated last year