A high-throughput and memory-efficient inference and serving engine for LLMs
☆47Oct 27, 2026Updated last week
Alternatives and similar repositories for vllm
Users that are interested in vllm are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- ☆17Jun 17, 2026Updated 4 months ago
- Archived: b12x development has moved to FlashInfer. See README for migration and PR-porting guidance.☆270Oct 6, 2026Updated last month
- Docker images for LLM inference (SGLang + vLLM) on NVIDIA Blackwell GPUs (SM120, CUDA 13.2)☆98Oct 20, 2026Updated 2 weeks ago
- ☆791Updated this week
- LLM inference decode throughput benchmark with Rich TUI dashboard. Measures token generation speed across concurrency levels and context …☆126Updated this week
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- Native DeepSeek V4.1 Flash inference across one RTX PRO 6000 and four DGX Spark expert workers☆40Sep 25, 2026Updated last month
- Low-latency collective transport and distributed LLM serving stack for switchless GB10 clusters.☆103Updated this week
- RTX 6000 Pro Wiki — Running Large LLMs (Qwen3.5-397B, Kimi-K2.5, GLM-5) on PCIe GPUs without NVLink☆1,193Updated this week
- Reproducible SGLang image for DeepSeek-V4-Flash-0731 on SM120☆38Aug 25, 2026Updated 2 months ago
- GLM-5.2 (744B/40B MoE) on a 4× DGX Spark / GB10 (sm_121) cluster: portable Triton sparse-MLA kernels, a data-free expert prune, MTP draft…☆37Aug 20, 2026Updated 2 months ago
- FlashQLA TileLang GDN kernels ported to NVIDIA Blackwell consumer (GB10 / DGX Spark)☆18Jun 5, 2026Updated 5 months ago
- GLM-5.3-Flash EXL3 (320B MoE) on 2x NVIDIA DGX Spark — production serving kit, 1M context, 97%+ multi-session prefix caching, DFlash2 spe…☆65Updated this week
- NVFP4 KV cache for vLLM on SM120 (RTX PRO 6000) via FlashInfer FA2 explicit-SF-stride patch — ~1.5x fp8 pool at ~95-104% speed☆20Jun 5, 2026Updated 5 months ago
- Give DeepSeek V4 Flash eyes. OpenAI-compatible shim + tiny local VLM. Point any harness at one URL and your text-only model accepts image…☆30Aug 9, 2026Updated 2 months ago
- Deploy open-source AI quickly and easily - Special Bonus Offer • AdRunpod Hub is built for open source. One-click deployment and autoscaling endpoints without provisioning your own infrastructure.
- DeepSeek 1017 EXL3 on 3-4 DGX Sparks. TP4 4.0.4: 4,277 tok/s cold prefill; C16 code 331.7 / prose 287.5 native decode sum. Smart CUDA Gra…☆19Updated this week
- DeepSeek-V4-Flash-0731 tuned on 4x DGX Spark (GB10) at TP=4 — 123.13 tok/s decode. Every variable measured one at a time, with the values…☆13Aug 11, 2026Updated 2 months ago
- Autonomous self-improving 4x DGX Spark (GB10) MoA stack + LoRA loop (DSV4F router, Qwen3.6/Omni/TwoTower/Gemma). Hermes MoA routing, ~90%…☆24Aug 17, 2026Updated 2 months ago
- Reproducible Docker/vLLM deployment, tuning, and benchmarks for DeepSeek V4.1 Flash on a switchless three-node or four-node DGX Spark fab…☆146Updated this week
- Tool-calling quality benchmark for LLM serving stacks. 80+ deterministic scenarios testing multi-turn orchestration, safety boundaries, a…☆417Updated this week
- Truthful scheduler-step and streaming visualization for local LLM inference☆35Sep 6, 2026Updated 2 months ago
- SGLang for GLM-5.3-Flash on two or four RTX PRO 6000 Blackwell GPUs (SM120), with TP2 W4A16 and TP4 NVIDIA NVFP4 profiles.☆48Oct 11, 2026Updated 3 weeks ago
- Apple Neural Engine on Linux: verified gemm/softmax/attention/transformer-block primitives, driven from Python on an M1 under Omarchy Mac☆19Updated this week
- MiniMax M2.7 NVFP4 dual-GB10 Blackwell benchmark: vLLM FlashInfer-CUTLASS, public data, HTML canvas report, and Docker runtime.☆13May 24, 2026Updated 5 months ago
- Wordpress hosting with auto-scaling - Free Trial Offer • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- ☆14Jul 22, 2025Updated last year
- Optimized cross-platform expansion of autoresearch.☆18Mar 23, 2026Updated 7 months ago
- Open Imi is a open source claude desktop alternative for developers, engineers and tech teams to hack MCP's and agents to their own likin…☆10Nov 16, 2025Updated 11 months ago
- GLM-5.2 on 4x DGX Spark with adaptive MTP K2/K4/K5, FULL CUDA graphs, DCP2, 520K context, and a downloadable ARM64 runtime image.☆13Aug 8, 2026Updated 3 months ago
- Index / recipe book for every GLM-5.3 (and GLM-5.2 foundation) serving recipe on NVIDIA DGX Spark (GB10): Flash + big 743B, TP2 + TP4, ev…☆13Aug 29, 2026Updated 2 months ago
- Minimal read-only OS image for GB10 (DGX Spark / ASUS GX10) boxes☆97Nov 1, 2026Updated last week
- ☆10Jan 22, 2025Updated last year
- Testbench for llama.cpp llama-server☆15Aug 20, 2025Updated last year
- Optimized inference for the DGX Spark and spark clusters☆139Updated this week
- Managed Database hosting by DigitalOcean • AdPostgreSQL, MySQL, MongoDB, Kafka, Valkey, and OpenSearch available. Automatically scale up storage and focus on building your apps.
- GLM-5.3-Flash (EXL3 4bpw + DFlash2) on 2x NVIDIA DGX Spark: one-shot installer, 33-74 tok/s c1, 1.3M+ context, vision☆43Sep 2, 2026Updated 2 months ago
- Docker configuration for running VLLM on dual DGX Sparks☆2,479Updated this week
- Working recipe: MiniMax-M3 NVFP4 at tensor-parallel 3 across 3x DGX Spark (GB10/sm_121) with clean tool-calling. Includes the head-node O…☆16Jul 13, 2026Updated 3 months ago
- DGX Spark / GB10 vLLM Docker stack for large-model serving, presets, patches, and validation notes.☆59Nov 1, 2026Updated last week
- DeepSeek-V4-Flash W4A16-FP8 quantization on 8x H200 — patches, recipe, mission report☆14Sep 26, 2026Updated last month
- MCP server for Vikunja task management☆17Oct 2, 2026Updated last month
- DGX Spark / GB10 vLLM Docker stack for large-model serving, presets, patches, and validation notes.☆12Jun 29, 2026Updated 4 months ago