A vLLM patch + hand‑written SM120 SASS kernels: 2‑bit MoE experts + an FP4 "delta" cache that recovers precision — matching the official (NV)FP4 checkpoint's quality on consumer Blackwell cards
☆545Oct 22, 2026Updated this week
Alternatives and similar repositories for vLLM-Moet
Users that are interested in vLLM-Moet are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- RTX 6000 Pro Wiki — Running Large LLMs (Qwen3.5-397B, Kimi-K2.5, GLM-5) on PCIe GPUs without NVLink☆1,156Updated this week
- Open-source SM120 (Blackwell) SASS assembler — encode, schedule & assemble GPU machine code, including all tensor-core instructions.☆12Updated this week
- Qwen3.6/3.8-27B on one RTX 5090: 576K pooled context across 12 sessions at FP8-level quality from 5.75 bpw. EXL3 trellis weights, 4-bit t…☆28Updated this week
- NVFP4 inference on Blackwell GeForce (RTX 5090/5080/5070 Ti/RTX PRO 6000) — SM120 patches for vLLM + FlashInfer + CUTLASS. 175 tok/s on Q…☆27Apr 27, 2026Updated 5 months ago
- AI on consumer Blackwell — a field guide for the RTX 50-series (sm_120). Fixes, footguns, and honest measurement from one RTX 5090.☆26Sep 9, 2026Updated last month
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- Control panel for VLLM, Sglang, llama.cpp, exllamav3☆1,835Oct 12, 2026Updated last week
- CUDA Blackwell performance fork of antirez/ds4 (DeepSeek-V4-Flash): ~4x prefill, ~1.5x decode, continuous batching, weight server, DSpark☆76Sep 4, 2026Updated last month
- ☆41Aug 16, 2026Updated 2 months ago
- Mixed-capability LLM benchmark for DGX Spark — 57 scenarios, 10 domains, partial-credit grading, trial statistics☆172Oct 13, 2026Updated last week
- SGLang for GLM-5.3-Flash on two or four RTX PRO 6000 Blackwell GPUs (SM120), with TP2 W4A16 and TP4 NVIDIA NVFP4 profiles.☆47Oct 11, 2026Updated 2 weeks ago
- Community registry of LLM serving-path traps that produce confidently wrong measurements: templates, tool parsers, reasoning fields, quan…☆140Oct 15, 2026Updated last week
- Teach GLM-5.2 to see art: minimal frozen-backbone vision-projector training (method by Harry Partridge)☆14Jul 18, 2026Updated 3 months ago
- Archived: b12x development has moved to FlashInfer. See README for migration and PR-porting guidance.☆269Oct 6, 2026Updated 2 weeks ago
- Docker images for LLM inference (SGLang + vLLM) on NVIDIA Blackwell GPUs (SM120, CUDA 13.2)☆93Updated this week
- Simple, predictable pricing with DigitalOcean hosting • AdAlways know what you'll pay with monthly caps and flat pricing. Enterprise-grade infrastructure trusted by 600k+ customers.
- A high-throughput and memory-efficient inference and serving engine for LLMs☆42Updated this week
- GLM-5.2 (744B/40B MoE) on a 4× DGX Spark / GB10 (sm_121) cluster: portable Triton sparse-MLA kernels, a data-free expert prune, MTP draft…☆37Aug 20, 2026Updated 2 months ago
- LLM speculative inference server for heterogeneous hardware & consumer GPUs☆2,953Updated this week
- ☆37Apr 26, 2026Updated 5 months ago
- AEON vLLM Ultimate — vLLM 0.27.1 built from source for DGX Spark / Blackwell (sm_121a/GB10). DSpark quantized Markov heads, DFlash SWA on…☆147Oct 6, 2026Updated 2 weeks ago
- DeepSeek-V4-Flash-DSpark optimized vLLM 0.26.0 SM121 dual-GB10 evidence (NVFP4 KV, B12X, DSpark K6)☆31Aug 3, 2026Updated 2 months ago
- MoonEP: A Perfectly Balanced Expert Parallelism Library via Dynamic Redundant Experts☆1,192Oct 12, 2026Updated last week
- ☆117Aug 5, 2026Updated 2 months ago
- Reverse-engineered instruction set architecture database for NVIDIA SM120 (Blackwell) GPUs.☆24Updated this week
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- Validated GLM-5.3 Flash recipe for 2x NVIDIA RTX PRO 6000 Blackwell 96GB: 262K context, EXL3/TR3, adaptive MTP, tools, and vision.☆61Sep 20, 2026Updated last month
- Optimized SGLang runtime for Qwen3.8-27B FP8 with DFlash2 and Qwen3.8 Flash-Next NVFP4 with FR-Spec on one NVIDIA RTX PRO 6000 Blackwell …☆178Updated this week
- sparkDash ⚡ — Multi-DGX Spark Monitoring Dashboard☆760Updated this week
- Plug-and-play, dependency-free web command center for DGX Spark fleets (and any GPU boxes): live per-node GPU/CPU/mem/temps/power, option…☆48Oct 13, 2026Updated last week
- SNDR Core Engine (Genesis) — vLLM runtime patch-overlay for Qwen3.6 + Gemma4 on consumer NVIDIA (Ampere sm_86, 2× A5000/3090). Qwen3.6-35…☆133Sep 30, 2026Updated 3 weeks ago
- Production-ready vLLM deployment wrapper for Qwen3.6-27B (NVFP4) — self-hosted OpenAI-compatible inference☆60Jul 30, 2026Updated 2 months ago
- ☆19Aug 9, 2026Updated 2 months ago
- ☆21Updated this week
- A unified library of SOTA model optimization techniques like quantization, distillation, pruning, neural architecture search, speculative…☆12Oct 13, 2026Updated last week
- End-to-end encrypted email - Proton Mail • AdSpecial offer: 40% Off Yearly / 80% Off First Month. All Proton services are open source and independently audited for security.
- ☆16Updated this week
- Efficient and unified implementations for TopK-based sparse attention☆41Jul 10, 2026Updated 3 months ago
- GLM-5.2-NVFP4-REAP-469B serving on SM120 (4× RTX PRO 6000 Blackwell) — one-command vLLM launch recipe, 250K context, DeepSeek Sparse Atte…☆155Jun 19, 2026Updated 4 months ago
- High performance RMSNorm Implement by using SM Core Storage(Registers and Shared Memory)☆31Jan 22, 2026Updated 9 months ago
- Fully uncensored, capability-enhanced abliteration of Qwen3.6-27B. NVFP4 + z-lab DFlash speculative decoding (n=12) on the unified ghcr.i…☆484Oct 6, 2026Updated 2 weeks ago
- Fast LLM server for NVIDIA Blackwell: GeForce RTX 50 and RTX PRO. Qwen3.8-27B on one RTX 5090: 91-94 tok/s, 1.3x vLLM and 1.4x llama.cpp.…☆47Updated this week
- An optimized quantization and inference library for running LLMs locally on modern consumer-class GPUs☆1,744Updated this week