A high-throughput and memory-efficient inference and serving engine for LLMs
☆38Oct 23, 2026Updated this week
Alternatives and similar repositories for vllm
Users that are interested in vllm are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- A cmpunlocker fork with working p2p on SYS-4028GR-TRT2☆26Aug 8, 2026Updated 2 months ago
- 1Cat-vLLM on Intel Gaudi☆43Oct 12, 2026Updated last week
- The complete NUMA-optimized branch of the ktransformers project☆26Nov 3, 2025Updated 11 months ago
- vllm混合推理扩展插件，支持多NUMA混合推理，单卡推理Qwen3-Next模型可达1000+ prefill☆34Nov 7, 2025Updated 11 months ago
- glm-5.2 on too many 64 / 80 GB A100's☆22Sep 9, 2026Updated last month
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- DeepSeek-V4-Flash on Ampere SM 8.6 via vLLM (pyref kernel replacements)☆35Jul 2, 2026Updated 3 months ago
- 双 SM80 / CMP 170HX 的 Qwen3.8-Flash-Next 社区 vLLM：NVIDIA NVFP4、GDS、TEP2、MTP6，支持 1～10 并发，词表与 PCIe 通信优化。中英文部署与可构建源码。☆28Oct 15, 2026Updated last week
- DeepSeek-V4-Flash-0731 on 4x CMP 170HX (sm_80): 98 tok/s decode, ~5300 tok/s prefill. DSpark speculative decoding under pipeline parallel…☆118Aug 13, 2026Updated 2 months ago
- DeepSeek-V4.1-Flash (764B) on 4×CMP 170HX: PP4 + shadow KV + EXL3 2bpw + DSpark + 512k ctx + native vision — patches, recipes, acceptance…☆13Sep 29, 2026Updated 3 weeks ago
- Dogfooding vLLM backport for older GPUs.☆332Updated this week
- A fork of jasl/vllm (codex/ds4-sm120-min-enable) with CPU-GPU hybrid inference support for DeepSeek-V4 on SM120+.☆31Sep 10, 2026Updated last month
- ☆14Jun 3, 2026Updated 4 months ago
- 一个Turing专用vLLM分支，通过整合FlashInfer 0.6.18（PR #3526）与SM75后端门槛补丁（PR #47949），持续同步vLLM主线最新版本☆34Oct 11, 2026Updated last week
- Linux P2P unlock and performance fixes for NVIDIA CMP 170HX☆39Sep 22, 2026Updated last month
- GPU virtual machines on DigitalOcean Gradient AI • AdGet to production fast with high-performance AMD and NVIDIA GPUs you can spin up in seconds. The definition of operational simplicity.
- Pytorch implementation of DeMatch for CVPR'24 paper "DeMatch: Deep Decomposition of Motion Field for Two-View Correspondence Learning"☆13Jun 5, 2024Updated 2 years ago
- GLM-5.3-Flash on 4× NVIDIA CMP 170HX with vLLM: up to 482.6 tok/s single-user, up to 943.7 tok/s at 8 users, 262K context, TP4 or PP4, on…☆84Oct 16, 2026Updated last week
- Tuning and qualification harness for the NVIDIA CMP 170HX (GA100). Measures, gates and recovers clock/voltage settings, and treats a comp…☆90Sep 28, 2026Updated 3 weeks ago
- pyqt package for blizzard like ui☆13Nov 28, 2016Updated 9 years ago
- Generates kid-friendly weather forecasts suitable for display on a large monitor. Uses OpenWeatherMap and a local or hosted LLM.☆16Feb 28, 2026Updated 7 months ago
- 在线曲库的B站直播弹幕点歌机☆12Apr 26, 2023Updated 3 years ago
- ☆64Aug 6, 2026Updated 2 months ago
- CalibRCNN: Calibrating Camera and LiDAR by Recurrent Convolutional Neural Network and Geometric Constraints☆11May 24, 2021Updated 5 years ago
- Automatically exported from code.google.com/p/linux-ftools☆16Apr 22, 2016Updated 10 years ago
- Managed Kubernetes at scale on DigitalOcean • AdDigitalOcean Kubernetes includes the control plane, bandwidth allowance, container registry, automatic updates, and more for free.
- Agnost GitOps Helm charts☆13Dec 9, 2025Updated 10 months ago
- PyTorch Implementation of Instant-NGP☆10Jan 9, 2023Updated 3 years ago
- 🤗 Transformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.☆10Nov 19, 2024Updated last year
- 使用nvidia oam转接卡将intel gaudi2转接到pcie并在ubuntu下正常驱动和使用的指南☆22Oct 10, 2026Updated 2 weeks ago
- ☆13Dec 30, 2020Updated 5 years ago
- FastAPI WebSocket server for the OpenVoice text-to-speech model.☆12Jun 6, 2024Updated 2 years ago
- Tries to UI development. Clone of https://www.perplexity.ai/☆11Sep 30, 2023Updated 3 years ago
- Want to work at Teambition? Check this to help you survive at Teambition.☆10Apr 1, 2016Updated 10 years ago
- Use OpenCode in WeChat☆23Mar 22, 2026Updated 7 months ago
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- 两个自己写的微信工具, 一个公众号内容抓取分类二次展示, 一个公众号后端服务, 均已部署在SAE上, 并取得日最高30万UV☆21Aug 6, 2017Updated 9 years ago
- ☆14Jan 12, 2024Updated 2 years ago
- Manage pasteboard history☆20Mar 7, 2021Updated 5 years ago
- Qwen3.8-Flash-Next 180B NVFP4 @ 231 tok/s on one RTX PRO 6000 Blackwell (sm120) - sglang patches, launchers, full optimization analysis☆30Aug 31, 2026Updated last month
- CPU-GPU hybrid inference for DeepSeek-V4 on NVIDIA SM80+ (A100/RTX 4090 etc.), forked from yhfgyyf/vllm-deepseek-v4-sm89.☆60Sep 16, 2026Updated last month
- Spartan-7 FPGA development board.☆11Apr 19, 2022Updated 4 years ago
- Simple photo browser written in Swift☆16Nov 4, 2020Updated 5 years ago