Dogfooding vLLM backport for older GPUs.
☆345Oct 27, 2026Updated this week
Alternatives and similar repositories for vllm-backport
Users that are interested in vllm-backport are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- A high-throughput and memory-efficient inference and serving engine for LLMs☆38Updated this week
- DeepSeek-V4-Flash-0731 on 4x CMP 170HX (sm_80): 98 tok/s decode, ~5300 tok/s prefill. DSpark speculative decoding under pipeline parallel…☆118Aug 13, 2026Updated 2 months ago
- DeepSeek-V4-Flash on Ampere SM 8.6 via vLLM (pyref kernel replacements)☆35Jul 2, 2026Updated 3 months ago
- 双 SM80 / CMP 170HX 的 Qwen3.8-Flash-Next 社区 vLLM：NVIDIA NVFP4、GDS、TEP2、MTP6，支持 1～10 并发，词表与 PCIe 通信优化。中英文部署与可构建源码。☆28Oct 15, 2026Updated 2 weeks ago
- Run DeepSeek-V4.1-Flash / DeepSeek-V4-Flash and GLM-5.3-Flash on SM89 (Ada / RTX 4090) and SM120 (RTX PRO 6000) with vLLM☆198Updated this week
- Serverless GPU API endpoints on Runpod - Get Bonus Credits • AdSkip the infrastructure headaches. Auto-scaling, pay-as-you-go, no-ops approach lets you focus on innovating your application.
- Community-tested recipes, diagnostics, and benchmarks for CMP 170HX AI workloads☆90Oct 19, 2026Updated last week
- CPU-GPU hybrid inference for DeepSeek-V4 on NVIDIA SM80+ (A100/RTX 4090 etc.), forked from yhfgyyf/vllm-deepseek-v4-sm89.☆61Sep 16, 2026Updated last month
- A tool to unlobotomize your NVIDIA card!☆31Sep 6, 2026Updated last month
- DeepSeek-V4.1-Flash inference on NVIDIA Ampere with vLLM, SM80 patches, reproducible throughput and coding-agent benchmarks. Tested on 8x…☆19Sep 15, 2026Updated last month
- DeepSeek-V4.1-Flash (764B) on 4×CMP 170HX: PP4 + shadow KV + EXL3 2bpw + DSpark + 512k ctx + native vision — patches, recipes, acceptance…☆15Sep 29, 2026Updated last month
- Linux P2P unlock and performance fixes for NVIDIA CMP 170HX☆39Sep 22, 2026Updated last month
- Async PLE SSD offload for Qwen3.8 Flash Next on a 64 GiB SM80 GPU: vLLM patch, setup guide, and measured PP/TG results.☆22Sep 24, 2026Updated last month
- ☆67Updated this week
- GPUDirect Storage (GDS) for the NVIDIA CMP 170HX on modern Linux kernels (6.18+). nvidia-fs NVMe registration hooks for the new blk_rq_dm…☆18Aug 19, 2026Updated 2 months ago
- AI Agents on DigitalOcean Gradient AI Platform • AdBuild production-ready AI agents using customizable tools or access multiple LLMs through a single endpoint. Create custom knowledge bases or connect external data.
- Qwen3.8-Flash-Next (W4A16) on 2x CMP 170HX: 110 tok/s decode, 7K prefill, 800 tok/s @ 23 concurrent - runs on 38 GB home RAM, no pinned m…☆20Sep 18, 2026Updated last month
- Optimized VLLM for Ampere☆24Oct 18, 2026Updated 2 weeks ago
- SGLang for cool dudes using V100☆82Oct 5, 2026Updated 3 weeks ago
- 1Cat-vLLM on Intel Gaudi☆44Oct 12, 2026Updated 2 weeks ago
- vLLM 0.32 fork for consumer GPUs: 3 x 262K sessions on 4x RTX 3090, Qwen 3.8 Flash-Next at 184 tok/s, up to 17 on bigger cards. Seven che…☆31Oct 22, 2026Updated last week
- A tool to unlobotomize your NVIDIA card!☆818Updated this week
- Unlock P2P comms between consumer NVIDIA GPUs☆684Updated this week
- V100 / SM70-focused vLLM engineering fork for modern LLM inference.☆1,431Updated this week
- 限制问题区域显存 62906MiB / 65024MiB 支持驱动 610.57.04 ，修改显卡显示名称为NVIDIA Tesla A100☆13Aug 30, 2026Updated 2 months ago
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- ☆36Oct 14, 2026Updated 2 weeks ago
- Qwen3.8-27B INT8 + DFlash2 on a rented CMP 170HX — failure analysis (private GHCR image prevented benchmark)☆17Aug 30, 2026Updated 2 months ago
- 一个 PPT 生成 Agent：在网页填写主题与约束，后端用 @opencode-ai/sdk 驱动 LLM 先产出可编辑大纲，再生成 HTML slides，最后本地构建 deck.pptx。☆14Mar 19, 2026Updated 7 months ago
- Lsglang is a special extension of sglang that fully utilizes CPU and GPU computing resources with an efficient GPU parallel + NUMA parall…☆164Updated this week
- Cross-tier CPU/GPU hybrid MoE inference for SGLang (plugin, tracks upstream main)☆47Updated this week
- Multi-lingual large voice generation model, providing inference, training and deployment full-stack ability.☆17Apr 15, 2026Updated 6 months ago
- RDMA library in Go☆18Jun 17, 2026Updated 4 months ago
- ☆132Updated this week
- GLM-5.3-Flash on 4× NVIDIA CMP 170HX with vLLM: up to 482.6 tok/s single-user, up to 943.7 tok/s at 8 users, 262K context, TP4 or PP4, on…☆87Updated this week
- AI Agents on DigitalOcean Gradient AI Platform • AdBuild production-ready AI agents using customizable tools or access multiple LLMs through a single endpoint. Create custom knowledge bases or connect external data.
- SNDR Core Engine (Genesis) — vLLM runtime patch-overlay for Qwen3.6 + Gemma4 on consumer NVIDIA (Ampere sm_86, 2× A5000/3090). Qwen3.6-35…☆133Sep 30, 2026Updated last month
- ☆13Oct 27, 2025Updated last year
- 使用nvidia oam转接卡将intel gaudi2转接到pcie并在ubuntu下正常驱动和使用的指南☆25Oct 10, 2026Updated 3 weeks ago
- An optimized quantization and inference library for running LLMs locally on modern consumer-class GPUs☆1,772Updated this week
- A ComfyUI custom node enabling **Flash Attention 1** on legacy NVIDIA GPUs (Tesla V100, T4) that lack Compute Capability 8.0+ required by…☆33Feb 9, 2026Updated 8 months ago
- MiniMax shipped Kimi's office skill codebase with find-and-replace. 13 byte-identical files, hardcoded 'kimi' in source, DLL compiled fro…☆21Feb 17, 2026Updated 8 months ago
- Wi-Fi enabled XVC programmer/debugger based on ESP8266☆12May 31, 2015Updated 11 years ago