Frontier MoE models on RTX PRO 6000 / RTX 5090 (sm_120): DeepSeek-V4.1-Flash on 4x RTX PRO 6000, as a GPU+CPU hybrid on 4x RTX 5090, and 2-bit experts on vLLM v0.24.0
☆577Nov 13, 2026Updated this week
Alternatives and similar repositories for vLLM-Moet
Users that are interested in vLLM-Moet are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- RTX 6000 Pro Wiki — Running Large LLMs (Qwen3.5-397B, Kimi-K2.5, GLM-5) on PCIe GPUs without NVLink☆1,206Updated this week
- Personal fork of Kacper’s vLLM-Moet for DeepSeek V4 Flash on a single RTX PRO 6000. Shared for others to use; support is best-effort as t…☆18Aug 19, 2026Updated 2 months ago
- Open-source SM120 (Blackwell) SASS assembler — encode, schedule & assemble GPU machine code, including all tensor-core instructions.☆14Updated this week
- AI on consumer Blackwell — a field guide for the RTX 50-series (sm_120). Fixes, footguns, and honest measurement from one RTX 5090.☆26Sep 9, 2026Updated 2 months ago
- Control panel for VLLM, Sglang, llama.cpp, exllamav3☆1,845Oct 26, 2026Updated 2 weeks ago
- Wordpress hosting with auto-scaling - Free Trial Offer • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- Qwen3.6/3.8-27B on one RTX 5090: 576K pooled context across 12 sessions at FP8-level quality from 5.75 bpw. EXL3 trellis weights, 4-bit t…☆30Oct 23, 2026Updated 3 weeks ago
- Docker images for LLM inference (SGLang + vLLM) on NVIDIA Blackwell GPUs (SM120, CUDA 13.2)☆100Oct 20, 2026Updated 3 weeks ago
- NVFP4 inference on Blackwell GeForce (RTX 5090/5080/5070 Ti/RTX PRO 6000) — SM120 patches for vLLM + FlashInfer + CUTLASS. 175 tok/s on Q…☆29Apr 27, 2026Updated 6 months ago
- ☆41Aug 16, 2026Updated 3 months ago
- Mixed-capability LLM benchmark for DGX Spark — 57 scenarios, 10 domains, partial-credit grading, trial statistics☆171Oct 13, 2026Updated last month
- SGLang for GLM-5.3-Flash on two or four RTX PRO 6000 Blackwell GPUs (SM120), with TP2 W4A16 and TP4 NVIDIA NVFP4 profiles.☆48Oct 11, 2026Updated last month
- Lucebox inference engine for heterogeneous hardware & consumer GPUs☆3,007Updated this week
- Archived: b12x development has moved to FlashInfer. See README for migration and PR-porting guidance.☆269Oct 6, 2026Updated last month
- Teach GLM-5.2 to see art: minimal frozen-backbone vision-projector training (method by Harry Partridge)☆14Jul 18, 2026Updated 3 months ago
- Wordpress hosting with auto-scaling - Free Trial Offer • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- Qwen3.8-Flash-Next optimized for 1x RTX Pro 6000☆20Sep 17, 2026Updated last month
- Community registry of LLM serving-path traps that produce confidently wrong measurements: templates, tool parsers, reasoning fields, quan…☆174Nov 7, 2026Updated last week
- SGLang 0.5.21 for Qwen3.8-Flash-Next on one RTX PRO 6000: 264.5 tok/s decode, W4A16 routed experts (KLD 0.0111 vs BF16), 262K context, ex…☆18Oct 26, 2026Updated 3 weeks ago
- ☆37Apr 26, 2026Updated 6 months ago
- Fast, capable GLM-5.3-Flash FP4-Local on two DGX Sparks: opinionated Atlas serving and reproducible inference research.☆73Updated this week
- Reverse-engineered instruction set architecture database for NVIDIA SM120 (Blackwell) GPUs.☆30Updated this week
- DeepSeek V4.1 Flash on 4x RTX PRO 6000 Blackwell: native weights, DSpark, vision, NVMe or locked RAM Engram offload.☆91Oct 17, 2026Updated 3 weeks ago
- GLM-5.2 744B with a true 4-bit NVFP4 KV cache on 4x DGX Spark: 42 tok/s peak, 317K-token KV pool (+58.6% vs fp8), needle-verified at 250K…☆15Jul 21, 2026Updated 3 months ago
- LLM inference decode throughput benchmark with Rich TUI dashboard. Measures token generation speed across concurrency levels and context …☆129Nov 4, 2026Updated last week
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- Btop-style ESP32-S3 touchscreen dashboard for vLLM and TensorFold inference servers☆13Updated this week
- ☆807Updated this week
- DeepSeek-V4-Flash-DSpark optimized vLLM 0.26.0 SM121 dual-GB10 evidence (NVFP4 KV, B12X, DSpark K6)☆31Aug 3, 2026Updated 3 months ago
- Optimized SGLang runtime for Qwen3.8-27B FP8 with DFlash2 and Qwen3.8 Flash-Next NVFP4 with FR-Spec on one NVIDIA RTX PRO 6000 Blackwell …☆200Updated this week
- A community-owned map of LLM inference engine configurations — benchmarks, evals and coverage gaps, hosted on GitHub Pages. The repo is t…☆16Oct 19, 2026Updated 3 weeks ago
- AEON vLLM Ultimate — vLLM 0.27.1 built from source for DGX Spark / Blackwell (sm_121a/GB10). DSpark quantized Markov heads, DFlash SWA on…☆149Updated this week
- An optimized quantization and inference library for running LLMs locally on modern consumer-class GPUs☆1,829Updated this week
- SNDR Core Engine (Genesis) — vLLM runtime patch-overlay for Qwen3.6 + Gemma4 on consumer NVIDIA (Ampere sm_86, 2× A5000/3090). Qwen3.6-35…☆133Sep 30, 2026Updated last month
- Production-ready vLLM deployment wrapper for Qwen3.6-27B (NVFP4) — self-hosted OpenAI-compatible inference☆60Jul 30, 2026Updated 3 months ago
- Simple, predictable pricing with DigitalOcean hosting • AdAlways know what you'll pay with monthly caps and flat pricing. Enterprise-grade infrastructure trusted by 600k+ customers.
- ☆11Jul 10, 2026Updated 4 months ago
- CUDA Blackwell performance fork of antirez/ds4 (DeepSeek-V4-Flash): ~4x prefill, ~1.5x decode, continuous batching, weight server, DSpark☆77Oct 26, 2026Updated 3 weeks ago
- MoonEP: A Perfectly Balanced Expert Parallelism Library via Dynamic Redundant Experts☆1,212Oct 12, 2026Updated last month
- High performance RMSNorm Implement by using SM Core Storage(Registers and Shared Memory)☆31Jan 22, 2026Updated 9 months ago
- A high-throughput and memory-efficient inference and serving engine for LLMs☆47Oct 27, 2026Updated 2 weeks ago
- Unlock P2P comms between consumer NVIDIA GPUs☆729Oct 26, 2026Updated 2 weeks ago
- Validated GLM-5.3 Flash recipe for 2x NVIDIA RTX PRO 6000 Blackwell 96GB: 262K context, EXL3/TR3, adaptive MTP, tools, and vision.☆63Sep 20, 2026Updated last month
Are these results useful?