Dogfooding vLLM backport for older GPUs.
☆365Nov 17, 2026Updated this week
Alternatives and similar repositories for vllm-backport
Users that are interested in vllm-backport are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- A high-throughput and memory-efficient inference and serving engine for LLMs☆38Nov 4, 2026Updated 2 weeks ago
- DeepSeek-V4-Flash on Ampere SM 8.6 via vLLM (pyref kernel replacements)☆36Jul 2, 2026Updated 4 months ago
- Run DeepSeek-V4.1-Flash / DeepSeek-V4-Flash and GLM-5.3-Flash on SM89 (Ada / RTX 4090) and SM120 (RTX PRO 6000) with vLLM☆209Updated this week
- DeepSeek-V4-Flash-0731 on 4x CMP 170HX (sm_80): 98 tok/s decode, ~5300 tok/s prefill. DSpark speculative decoding under pipeline parallel…☆120Aug 13, 2026Updated 3 months ago
- glm-5.2 on too many 64 / 80 GB A100's☆22Sep 9, 2026Updated 2 months ago
- Virtual machines for every use case on DigitalOcean • AdGet dependable uptime with 99.99% SLA, simple security tools, and predictable monthly pricing with DigitalOcean's virtual machines, called Droplets.
- 在四卡A100上适配Sglang部署deepseek-v4-flash正式版模型☆15Aug 2, 2026Updated 3 months ago
- CPU-GPU hybrid inference for DeepSeek-V4 on NVIDIA SM80+ (A100/RTX 4090 etc.), forked from yhfgyyf/vllm-deepseek-v4-sm89.☆61Sep 16, 2026Updated 2 months ago
- vLLM for NVIDIA CMP 170HX (Ampere sm_80): fast, repeatable model serving on one card or several☆32Nov 2, 2026Updated 2 weeks ago
- 双 SM80 / CMP 170HX 的 Qwen3.8-Flash-Next 社区 vLLM:NVIDIA NVFP4、GDS、TEP2、MTP6,支持 1~10 并发,词表与 PCIe 通信优化。中英文部署与可构建源码。☆28Oct 15, 2026Updated last month
- Qwen3.8-Flash-Next-FP8 (official checkpoint) on 3x/4x CMP170HX / PP=4 / MTP-4☆12Updated this week
- ☆70Nov 4, 2026Updated 2 weeks ago
- Dogfooding vLLM backport for older GPUs.☆14Oct 29, 2026Updated 3 weeks ago
- Linux P2P unlock and performance fixes for NVIDIA CMP 170HX☆39Sep 22, 2026Updated last month
- Community-tested recipes, diagnostics, and benchmarks for CMP 170HX AI workloads☆101Nov 9, 2026Updated last week
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- DeepSeek-V4.1-Flash on 4× NVIDIA CMP 170HX with vLLM: up to 297 tok/s for one user, up to 868 tok/s at eight, 262K context, one-command c…☆37Nov 2, 2026Updated 2 weeks ago
- A tool to unlobotomize your NVIDIA card!☆31Sep 6, 2026Updated 2 months ago
- DeepSeek-V4.1-Flash (764B) on 4×CMP 170HX: PP4 + shadow KV + EXL3 2bpw + DSpark + 512k ctx + native vision — patches, recipes, acceptance…☆15Sep 29, 2026Updated last month
- dsv41-flash-pp5-170hx &pp6 Supports the original (full/unquantized) model. Prefill 7,000 t/s, decode 45-75-125 t/s. dspark. Requires 200 …☆13Updated this week
- GPUDirect Storage (GDS) for the NVIDIA CMP 170HX on modern Linux kernels (6.18+). nvidia-fs NVMe registration hooks for the new blk_rq_dm…☆19Aug 19, 2026Updated 3 months ago
- Qwen3.8 Flash Next AWQ INT4 4x3090 v1: 262K, Vision, FP8 QSA KV and CUDA Graphs on 4x RTX 3090☆16Aug 30, 2026Updated 2 months ago
- NInfer SM80 port for NVIDIA CMP 170HX☆29Aug 22, 2026Updated 2 months ago
- ☆36Oct 20, 2026Updated last month
- Optimized VLLM for Ampere☆24Oct 18, 2026Updated last month
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- Qwen3.8-Flash-Next (W4A16) on 2x CMP 170HX: 110 tok/s decode, 7K prefill, 800 tok/s @ 23 concurrent - runs on 38 GB home RAM, no pinned m…☆20Sep 18, 2026Updated 2 months ago
- Lsglang is a special extension of sglang that fully utilizes CPU and GPU computing resources with an efficient GPU parallel + NUMA parall…☆169Updated this week
- Unlock P2P comms between consumer NVIDIA GPUs☆744Oct 26, 2026Updated 3 weeks ago
- Fast vLLM on AMD Strix Halo: one docker command for Qwen, Gemma, LFM and more, up to 31x faster prefill and ~2x decode vs stock vLLM at l…☆22Nov 8, 2026Updated last week
- LvLLM is a special NUMA extension of vllm that makes full use of CPU and memory resources, reduces GPU memory requirements, and features …☆476Nov 6, 2026Updated 2 weeks ago
- The complete NUMA-optimized branch of the ktransformers project☆27Nov 3, 2025Updated last year
- SGLang for cool dudes using V100☆87Nov 11, 2026Updated last week
- GLM-5.3-Flash on 4× NVIDIA CMP 170HX with vLLM: up to 482.6 tok/s single-user, up to 943.7 tok/s at 8 users, 262K context, TP4 or PP4, on…☆93Oct 26, 2026Updated 3 weeks ago
- A tool to unlobotomize your NVIDIA card!☆940Oct 28, 2026Updated 3 weeks ago
- Wordpress hosting with auto-scaling - Free Trial Offer • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- XTU = X Transformers Unity — hybrid (CPU-expert + GPU) inference acceleration for vLLM: mainline plugin + CPU MoE engine (AVX512-VNNI/BF1…☆16Nov 2, 2026Updated 2 weeks ago
- NVIDIA Linux open GPU with P2P support☆48Oct 26, 2026Updated 3 weeks ago
- V100 / SM70-focused vLLM engineering fork for modern LLM inference.☆1,550Updated this week
- CMP 170HX technical wiki: silicon, firmware, the community unlock, procedures and open problems. 55 pages.☆80Aug 22, 2026Updated 2 months ago
- ☆37Updated this week
- A cmpunlocker fork with working p2p on SYS-4028GR-TRT2☆27Aug 8, 2026Updated 3 months ago
- FORK of VLLM for AMD MI25/50/60. A high-throughput and memory-efficient inference and serving engine for LLMs☆71May 4, 2025Updated last year
Are these results useful?