Run DeepSeek-V4.1-Flash / DeepSeek-V4-Flash and GLM-5.3-Flash on SM89 (Ada / RTX 4090) and SM120 (RTX PRO 6000) with vLLM
☆209Nov 18, 2026Updated this week
Alternatives and similar repositories for vllm-deepseek-v4-sm89
Users that are interested in vllm-deepseek-v4-sm89 are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- CPU-GPU hybrid inference for DeepSeek-V4 on NVIDIA SM80+ (A100/RTX 4090 etc.), forked from yhfgyyf/vllm-deepseek-v4-sm89.☆61Sep 16, 2026Updated 2 months ago
- Dogfooding vLLM backport for older GPUs.☆365Updated this week
- ☆11Updated this week
- 1Cat-vLLM on Intel Gaudi☆48Oct 12, 2026Updated last month
- LvLLM is a special NUMA extension of vllm that makes full use of CPU and memory resources, reduces GPU memory requirements, and features …☆476Nov 6, 2026Updated 2 weeks ago
- Managed Kubernetes at scale on DigitalOcean • AdDigitalOcean Kubernetes includes the control plane, bandwidth allowance, container registry, automatic updates, and more for free.
- NVIDIA Linux open GPU with P2P support☆48Oct 26, 2026Updated 3 weeks ago
- Qwen3.8-Flash-Next 180B NVFP4 @ 231 tok/s on one RTX PRO 6000 Blackwell (sm120) - sglang patches, launchers, full optimization analysis☆30Aug 31, 2026Updated 2 months ago
- Lsglang is a special extension of sglang that fully utilizes CPU and GPU computing resources with an efficient GPU parallel + NUMA parall…☆169Updated this week
- Qwen3.8-27B on one RTX 4090: full native 262K context via E8 4-bit KV, up to 149 tok/s code decode with MTP3, sm_89-retuned attention pre…☆15Sep 29, 2026Updated last month
- DeepSeek-V4-Flash on Ampere SM 8.6 via vLLM (pyref kernel replacements)☆36Jul 2, 2026Updated 4 months ago
- vllm混合推理扩展插件,支持多NUMA混合推理,单卡推理Qwen3-Next模型可达1000+ prefill☆34Nov 7, 2025Updated last year
- Local LLM Inference Speed Test Tool☆256Oct 21, 2026Updated last month
- The complete NUMA-optimized branch of the ktransformers project☆27Nov 3, 2025Updated last year
- Cross-tier CPU/GPU hybrid MoE inference for SGLang (plugin, tracks upstream main)☆85Updated this week
- Managed Kubernetes at scale on DigitalOcean • AdDigitalOcean Kubernetes includes the control plane, bandwidth allowance, container registry, automatic updates, and more for free.
- FORK of VLLM for AMD MI25/50/60. A high-throughput and memory-efficient inference and serving engine for LLMs☆71May 4, 2025Updated last year
- Run AMD AITER's hand-written ASM kernels on CDNA2 / gfx90a (MI210, MI250). 242 of 1,422 kernels translated, with the tests and benchmarks…☆11Sep 9, 2026Updated 2 months ago
- Qwen3.8-Flash-Next (W4A16) on 2x NVIDIA CMP 170HX with vLLM pipeline parallel: 226 tok/s single-user, 840 tok/s at 8 users, 262K context,…☆12Nov 11, 2026Updated last week
- 把 Codex 原版 computer-use 运行时做成一个普通命令行工具。☆16Oct 28, 2026Updated 3 weeks ago
- Guide to using an NVIDIA OAM adapter card to connect an Intel Gaudi2 to PCIe and get it properly driven and working under Ubuntu☆31Nov 4, 2026Updated 2 weeks ago
- llama.cpp fork for two V100 32GB cards to run Qwen3.8FlashNext☆15Nov 3, 2026Updated 2 weeks ago
- MiniMax shipped Kimi's office skill codebase with find-and-replace. 13 byte-identical files, hardcoded 'kimi' in source, DLL compiled fro…☆22Feb 17, 2026Updated 9 months ago
- ☆15Oct 29, 2026Updated 3 weeks ago
- Unlock P2P comms between consumer NVIDIA GPUs☆744Oct 26, 2026Updated 3 weeks ago
- Proton VPN Special Offer - Get 70% off • AdSpecial partner offer. Trusted by over 100 million users worldwide. Tested, Approved and Recommended by Experts.
- Dogfooding vLLM backport for older GPUs.☆14Oct 29, 2026Updated 3 weeks ago
- fastllm是后端无依赖的高性能大模型推理库,帮助您在大部分设备上高效推理大模型!☆5,133Updated this week
- Framework to achieve context distillation in LLMs☆15Nov 24, 2023Updated 2 years ago
- V100 / SM70-focused vLLM engineering fork for modern LLM inference.☆1,550Updated this week
- Confucius4-R2T2 GGUF Q8 speech-to-text nodes for ComfyUI (Windows/CUDA)☆17Nov 2, 2026Updated 2 weeks ago
- ☆29Updated this week
- Tested recipes, runtime fixes, kernels and fleet practices for serving LLMs with vLLM on Intel Gaudi☆11Oct 25, 2026Updated 3 weeks ago
- dsv41-flash-pp5-170hx &pp6 Supports the original (full/unquantized) model. Prefill 7,000 t/s, decode 45-75-125 t/s. dspark. Requires 200 …☆13Updated this week
- ☆13Mar 27, 2026Updated 7 months ago
- Managed Kubernetes at scale on DigitalOcean • AdDigitalOcean Kubernetes includes the control plane, bandwidth allowance, container registry, automatic updates, and more for free.
- Single-card Intel Gaudi2 vLLM source snapshot: FP8 Qwen, non-greedy DFlash2/MTP, GDN kernels and host-side optimizations.☆20Updated this week
- GLM-5.3-Flash on 4× NVIDIA CMP 170HX with vLLM: up to 482.6 tok/s single-user, up to 943.7 tok/s at 8 users, 262K context, TP4 or PP4, on…☆93Oct 26, 2026Updated 3 weeks ago
- 适用于 Turing 架构显卡的 ComfyUI 轮子库 Linux 版☆12Jan 12, 2026Updated 10 months ago
- Xolotl: run DeepSeek-V4.1-Flash, all 552B, as a local coding agent on one gaming PC (RTX 4090 + 110 GB RAM): ~33 tok/s on agent tasks, ~4…☆545Updated this week
- DeepSeek-V4.1-Flash (764B) on 4×CMP 170HX: PP4 + shadow KV + EXL3 2bpw + DSpark + 512k ctx + native vision — patches, recipes, acceptance…☆15Sep 29, 2026Updated last month
- A prompt-aware LLM router that predicts which models can complete each request, then selects the cheapest capable one: 53.2% lower cost a…☆42Oct 26, 2026Updated 3 weeks ago
- Calibration-free OrbitQuant for transformer backbones, with Hugging Face integration and packed CUDA/Metal inference☆15Nov 11, 2026Updated last week
Are these results useful?