A high-throughput and memory-efficient inference and serving engine for LLMs
☆38Nov 4, 2026Updated last week
Alternatives and similar repositories for vllm
Users that are interested in vllm are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Dogfooding vLLM backport for older GPUs.☆362Updated this week
- vLLM for NVIDIA CMP 170HX (Ampere sm_80): fast, repeatable model serving on one card or several☆28Nov 2, 2026Updated last week
- The complete NUMA-optimized branch of the ktransformers project☆27Nov 3, 2025Updated last year
- vllm混合推理扩展插件，支持多NUMA混合推理，单卡推理Qwen3-Next模型可达1000+ prefill☆34Nov 7, 2025Updated last year
- Async PLE SSD offload for Qwen3.8 Flash Next on a 64 GiB SM80 GPU: vLLM patch, setup guide, and measured PP/TG results.☆23Sep 24, 2026Updated last month
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- A cmpunlocker fork with working p2p on SYS-4028GR-TRT2☆27Aug 8, 2026Updated 3 months ago
- DeepSeek-V4-Flash on Ampere SM 8.6 via vLLM (pyref kernel replacements)☆36Jul 2, 2026Updated 4 months ago
- This is a repository for documenting the setup and performance of MI100s in popular inference engines.☆21Oct 8, 2026Updated last month
- ☆28Oct 20, 2026Updated 3 weeks ago
- A (nicer) tokenizer you want to use for model inference and training: with all known peventable gotchas normalized or auto-fixed.☆12Oct 16, 2026Updated 3 weeks ago
- Fully chunked KL-Loss kernel, that massively reduces VRAM usage☆20Aug 10, 2026Updated 3 months ago
- DeepSeek-V4-Flash W4A16-FP8 quantization on 8x H200 — patches, recipe, mission report☆13Sep 26, 2026Updated last month
- Single-card Intel Gaudi2 vLLM source snapshot: FP8 Qwen, non-greedy DFlash2/MTP, GDN kernels and host-side optimizations.☆18Updated this week
- Code for STM32 on the UltraPIF☆14Feb 3, 2024Updated 2 years ago
- Wordpress hosting with auto-scaling - Free Trial Offer • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- high-performance linear attention kernel library built on TileLang☆15Jun 8, 2026Updated 5 months ago
- Experimental implementation of DeepSeek v4 flaash in llama.cpp☆23Apr 30, 2026Updated 6 months ago
- DeepSeek-V4.1-Flash (764B) on 4×CMP 170HX: PP4 + shadow KV + EXL3 2bpw + DSpark + 512k ctx + native vision — patches, recipes, acceptance…☆15Sep 29, 2026Updated last month
- Distillation pipeline from pretrained Transformers to customized FLA models☆13Aug 4, 2025Updated last year
- ☆17Oct 20, 2026Updated 3 weeks ago
- A clean and simple solution to replace the massive ribbon cable in the PCE and Core Grafx.☆16Aug 25, 2025Updated last year
- CuteDSL Swizzle Visualize Tool☆14Feb 22, 2026Updated 8 months ago
- Scripts to setup a two-node llama.cpp cluster on NVIDIA DGX Spark (GB10) hardware.☆12May 10, 2026Updated 6 months ago
- Verilog design for hardware ray-caster game☆11Aug 14, 2023Updated 3 years ago
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- GLM-5.3-Flash on 4× NVIDIA CMP 170HX with vLLM: up to 482.6 tok/s single-user, up to 943.7 tok/s at 8 users, 262K context, TP4 or PP4, on…☆90Oct 26, 2026Updated 2 weeks ago
- Spartan-7 FPGA development board.☆11Apr 19, 2022Updated 4 years ago
- An HDMI output device/project for the Gameboy (Color, Advance, etc.) using the Raspberry Pi Pico/RP2040.☆21Mar 27, 2022Updated 4 years ago
- GPU peer-to-peer over dynamic BAR1 for GeForce cards: patched NVIDIA open kernel modules.☆19Nov 6, 2026Updated last week
- ☆14Sep 4, 2024Updated 2 years ago
- Verilog code of Loongson's GS132 core☆12Dec 19, 2019Updated 6 years ago
- AMD PACE is a PyTorch extension for high-performance LLM inference on AMD CPUs. It provides a framework to develop and test novel optimiz…☆17Oct 21, 2026Updated 3 weeks ago
- Path tracing engine based on Taichi☆12Aug 3, 2021Updated 5 years ago
- Lower chisel memories to SRAM macros☆13Mar 25, 2024Updated 2 years ago
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- Use OpenCode in WeChat☆23Mar 22, 2026Updated 7 months ago
- ☆23Dec 15, 2016Updated 9 years ago
- A fork of jasl/vllm (codex/ds4-sm120-min-enable) with CPU-GPU hybrid inference support for DeepSeek-V4 on SM120+.☆32Sep 10, 2026Updated 2 months ago
- [CVPR 2026, CVEU] Official codebase for FOCUS: Optimal Control for Multi-Entity World Modeling in Text-to-Image Generation☆15Jun 9, 2026Updated 5 months ago
- Mechanical Squid Factory XInput PC Fight Stick☆31Apr 12, 2016Updated 10 years ago
- Tested recipes, runtime fixes, kernels and fleet practices for serving LLMs with vLLM on Intel Gaudi☆11Oct 25, 2026Updated 2 weeks ago
- Testing LLM reasoning abilities with lineage relationship quizzes.☆46Mar 10, 2026Updated 8 months ago
Are these results useful?