Run DeepSeek-V4.1-Flash / DeepSeek-V4-Flash and GLM-5.3-Flash on SM89 (Ada / RTX 4090) and SM120 (RTX PRO 6000) with vLLM
☆198Oct 28, 2026Updated this week
Alternatives and similar repositories for vllm-deepseek-v4-sm89
Users that are interested in vllm-deepseek-v4-sm89 are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Dogfooding vLLM backport for older GPUs.☆345Updated this week
- CPU-GPU hybrid inference for DeepSeek-V4 on NVIDIA SM80+ (A100/RTX 4090 etc.), forked from yhfgyyf/vllm-deepseek-v4-sm89.☆61Sep 16, 2026Updated last month
- DeepSeek-V4-Flash W4A16-FP8 quantization on 8x H200 — patches, recipe, mission report☆14Sep 26, 2026Updated last month
- 1Cat-vLLM on Intel Gaudi☆44Oct 12, 2026Updated 2 weeks ago
- vllm混合推理扩展插件，支持多NUMA混合推理，单卡推理Qwen3-Next模型可达1000+ prefill☆34Nov 7, 2025Updated 11 months ago
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- DeepSeek-V4-Flash on Ampere SM 8.6 via vLLM (pyref kernel replacements)☆35Jul 2, 2026Updated 3 months ago
- NVIDIA Linux open GPU with P2P support☆42Updated this week
- LvLLM is a special NUMA extension of vllm that makes full use of CPU and memory resources, reduces GPU memory requirements, and features …☆474Sep 22, 2026Updated last month
- Lsglang is a special extension of sglang that fully utilizes CPU and GPU computing resources with an efficient GPU parallel + NUMA parall…☆164Updated this week
- DeepSeek V4 Flash DSpark DGX Spark TP4 Ring 生产调优与算子工程归档 (LuZ0.4.5)☆17Sep 11, 2026Updated last month
- ☆43Sep 30, 2026Updated last month
- Exact SGLang runtime for GLM-5.3-Flash-NVFP4 on 4x RTX PRO 6000 Blackwell (SM120): flashinfer_cutlass MoE, DSA+KDA attention, NEXTN MTP5,…☆17Aug 27, 2026Updated 2 months ago
- Docker images for LLM inference (SGLang + vLLM) on NVIDIA Blackwell GPUs (SM120, CUDA 13.2)☆96Oct 20, 2026Updated last week
- RTX 6000 Pro Wiki — Running Large LLMs (Qwen3.5-397B, Kimi-K2.5, GLM-5) on PCIe GPUs without NVLink☆1,162Updated this week
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- 多引擎（FunASR / Qwen3）语音转录 + 说话人识别服务，WebSocket API，macOS / CUDA，支持音视频转写，分角色，自动合并相同人物的话。☆17Updated this week
- FORK of VLLM for AMD MI25/50/60. A high-throughput and memory-efficient inference and serving engine for LLMs☆71May 4, 2025Updated last year
- mc-brpc: A more convenient rpc framwork based on brpc to create rpc service☆15Feb 21, 2025Updated last year
- Local LLM Inference Speed Test Tool☆247Oct 21, 2026Updated last week
- forked from vllm-project/flash-attention☆70May 9, 2026Updated 5 months ago
- GLM-5.3-Flash on 4× NVIDIA CMP 170HX with vLLM: up to 482.6 tok/s single-user, up to 943.7 tok/s at 8 users, 262K context, TP4 or PP4, on…☆87Updated this week
- 跟手搓PPT说拜拜☆18Aug 22, 2026Updated 2 months ago
- Framework to achieve context distillation in LLMs☆15Nov 24, 2023Updated 2 years ago
- Fast & lightweight interface for running Nano Banana LLM and compact local models on your PC. Optimized for high-speed AI interactions.☆44Mar 17, 2026Updated 7 months ago
- GPU virtual machines on DigitalOcean Gradient AI • AdGet to production fast with high-performance AMD and NVIDIA GPUs you can spin up in seconds. The definition of operational simplicity.
- DeepSeek-V4.1-Flash (764B) on 4×CMP 170HX: PP4 + shadow KV + EXL3 2bpw + DSpark + 512k ctx + native vision — patches, recipes, acceptance…☆15Sep 29, 2026Updated last month
- Single-request LLM serving recipes, patches, and benchmarks for modified RTX 2080 Ti 22GB / SM75 systems☆97Jun 10, 2026Updated 4 months ago
- ☆17Sep 4, 2026Updated last month
- high-performance linear attention kernel library built on TileLang☆14Jun 8, 2026Updated 4 months ago
- ☆52Apr 27, 2026Updated 6 months ago
- A set of ComfyUI nodes for FlowUpscaler - a fast Flux.2 latent upscale model☆22Jun 13, 2026Updated 4 months ago
- Native DeepSeek V4.1 Flash inference across one RTX PRO 6000 and four DGX Spark expert workers☆38Sep 25, 2026Updated last month
- Cross-tier CPU/GPU hybrid MoE inference for SGLang (plugin, tracks upstream main)☆47Updated this week
- ☆21Oct 14, 2026Updated 2 weeks ago
- End-to-end encrypted email - Proton Mail • AdSpecial offer: 40% Off Yearly / 80% Off First Month. All Proton services are open source and independently audited for security.
- fastllm是后端无依赖的高性能大模型推理库，帮助您在大部分设备上高效推理大模型！☆5,130Updated this week
- ☆15Jan 16, 2026Updated 9 months ago
- marlin_v100 是一个从 vLLM 主树中提取出来的最小 Marlin 独立开发工作区，聚焦于 Marlin dense 与 Marlin MoE 的源码开发、最小构建和轻量验证。它保留了核心 CUDA/C++ 实现、最小 Python 薄封装、生成器测试与主树回写…☆26Jul 2, 2026Updated 3 months ago
- Numerical experiments on Jacobi SVD algorithm☆10Jun 3, 2018Updated 8 years ago
- This project is specifically developed for V100, based on lmdeploy 0.12.1, and supports mainstream open-source models from Q4 2025 to Q1 …☆22Mar 18, 2026Updated 7 months ago
- GLM-5.2 on 4x DGX Spark with adaptive MTP K2/K4/K5, FULL CUDA graphs, DCP2, 520K context, and a downloadable ARM64 runtime image.☆12Aug 8, 2026Updated 2 months ago
- Single-file, dependency-free live dashboard for local llama.cpp/vLLM serving — GPU, throughput, KV/ctx, model library. Stdlib Python + on…☆76Aug 15, 2026Updated 2 months ago