Serve EXL3 (ExLlamaV3 trellis) quantized models on vLLM fork runtimes — any architecture, mixed per-layer bitrates, composable with source-format non-routed weights
☆27Nov 5, 2026Updated this week
Alternatives and similar repositories for vllm-exl3
Users that are interested in vllm-exl3 are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Serve Qwen3.8-Flash-Next (turboderp ExLlamaV3 pack) on one NVIDIA DGX Spark with exllamav3☆64Oct 14, 2026Updated 3 weeks ago
- Async PLE SSD offload for Qwen3.8 Flash Next on a 64 GiB SM80 GPU: vLLM patch, setup guide, and measured PP/TG results.☆23Sep 24, 2026Updated last month
- Stackable, modular, 3D-printable rack for the NVIDIA DGX Spark. ABS/ASA, panels, CC BY-NC-SA 4.0.☆18Oct 8, 2026Updated 3 weeks ago
- Index / recipe book for every GLM-5.3 (and GLM-5.2 foundation) serving recipe on NVIDIA DGX Spark (GB10): Flash + big 743B, TP2 + TP4, ev…☆13Aug 29, 2026Updated 2 months ago
- Full GLM-5.3 (753B) served by TensorFold with tensor parallel 4 across four NVIDIA DGX Sparks: results, recipe, node settings, benches☆22Oct 13, 2026Updated 3 weeks ago
- End-to-end encrypted email - Proton Mail • AdSpecial offer: 40% Off Yearly / 80% Off First Month. All Proton services are open source and independently audited for security.
- Build, load, and verify NCCL for switchless DGX Spark RoCE cycles☆21Oct 25, 2026Updated last week
- EXLR8: a DGX Spark tuned quantization format☆14Updated this week
- DSV4.1-Flash-1017 SAGE EXL3 (1.62 bpw) on one DGX Spark with vLLM: DSpark MTP, cold-expert ATS alias, tile-K 16 MoE, memory guard, measur…☆40Updated this week
- Experimental runtime reclamation of GB10 display-reserved memory into ordinary Linux RAM, with guarded driver, local signing, and reboot …☆13Oct 23, 2026Updated 2 weeks ago
- Three-DGX-Spark TP3 serving recipe for GLM-5.3 Flash☆21Oct 5, 2026Updated last month
- ☆16Sep 29, 2026Updated last month
- ☆16Oct 20, 2026Updated 2 weeks ago
- GLM-5.2 UD-Q2 decode speedup: CPU∥GPU MoE expert-split (harness, specs, de-risk)☆23Aug 9, 2026Updated 2 months ago
- Pinned one-DGX-Spark Docker recipe for DeepSeek V4 Flash with EXL3, SparkInfer, and 262K NVFP4 MLA KV cache☆32Sep 6, 2026Updated 2 months ago
- Managed Kubernetes at scale on DigitalOcean • AdDigitalOcean Kubernetes includes the control plane, bandwidth allowance, container registry, automatic updates, and more for free.
- Native DeepSeek V4.1 Flash inference across one RTX PRO 6000 and four DGX Spark expert workers☆39Sep 25, 2026Updated last month
- A from-scratch CUDA inference engine for 1-6 DGX Sparks (GB10): GLM-5.3, GLM-5.3-Flash and DeepSeek-V4.1-Flash☆60Updated this week
- GLM-5.3-Flash (NVFP4) at TP=4 across 4x ASUS GX10 / GB10 - vLLM + mentat instead of Ray, 524k context, DFlash2. Minimal working recipe.☆94Oct 24, 2026Updated last week
- Serve DeepSeek-V4-Flash-Vision EXL3 MixedK (full 256 experts) on one NVIDIA DGX Spark with vLLM — recipe, loader patches, and receipts☆14Sep 7, 2026Updated last month
- A tiered-memory system design for workloads that don't fit in RAM: measure the working set, pin the hot tier, stream the cold tier from f…☆33Updated this week
- ☆37Apr 26, 2026Updated 6 months ago
- EKOS is an AI-native platform that continuously reconstructs, compiles, stores and serves enterprise knowledge.☆26Updated this week
- vLLM inference solutions for NVIDIA GB10 (DGX Spark, sm121) — KIMI-K3 B12X_MLA + DSpark, GLM-5.2☆68Sep 30, 2026Updated last month
- Full GLM-5.3 on 4x DGX Spark with kindlingai gr8 (TP4): RigMark prose c1 29.0 -> 40.6 tok/s (+40 %), c4 63.7 -> 76.6☆12Oct 25, 2026Updated last week
- Managed Database hosting by DigitalOcean • AdPostgreSQL, MySQL, MongoDB, Kafka, Valkey, and OpenSearch available. Automatically scale up storage and focus on building your apps.
- ☆26Oct 20, 2026Updated 2 weeks ago
- Fan Shroud for the Spark platform, for improved cooling performance on long workloads. Full STL/STEP files + electronics schematics☆41Sep 22, 2026Updated last month
- SGLang for GLM-5.3-Flash on two or four RTX PRO 6000 Blackwell GPUs (SM120), with TP2 W4A16 and TP4 NVIDIA NVFP4 profiles.☆48Oct 11, 2026Updated 3 weeks ago
- Qwen3.8 Flash-Next (NVFP4) on NVIDIA Flare for one DGX Spark / ASUS GX10: greedy, 8-agent speculative decoding, mixed prefill☆14Updated this week
- Fast, lightweight and stealth Firecrawl alternative in Rust. Web scraper, crawler & search API with MCP server for AI agents. Drop-in Fir…☆79Oct 18, 2026Updated 2 weeks ago
- vLLM for NVIDIA CMP 170HX (Ampere sm_80): fast, repeatable model serving on one card or several☆27Updated this week
- MiniMax M2.7 NVFP4 dual-GB10 Blackwell benchmark: vLLM FlashInfer-CUTLASS, public data, HTML canvas report, and Docker runtime.☆13May 24, 2026Updated 5 months ago
- Single-request LLM serving recipes, patches, and benchmarks for modified RTX 2080 Ti 22GB / SM75 systems☆97Jun 10, 2026Updated 4 months ago
- A simple PCB adapter for use on a SuperFX or CX4 donor cart☆12Sep 21, 2025Updated last year
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- DeepSeek-V4.1-Flash-1017 EXL3 on 2x DGX Sparks with TensorFold☆13Updated this week
- AEON vLLM Ultimate — vLLM 0.27.1 built from source for DGX Spark / Blackwell (sm_121a/GB10). DSpark quantized Markov heads, DFlash SWA on…☆148Oct 6, 2026Updated last month
- Validated GLM-5.3 Flash recipe for 2x NVIDIA RTX PRO 6000 Blackwell 96GB: 262K context, EXL3/TR3, adaptive MTP, tools, and vision.☆62Sep 20, 2026Updated last month
- Code for STM32 on the UltraPIF☆13Feb 3, 2024Updated 2 years ago
- We gave AI models telepathy.☆22Sep 25, 2026Updated last month
- Boo, a ghost-faced audio/visual agent you talk to from your phone: speech in and out, camera, hand tracking, memory and a tool-using back…☆14Oct 30, 2026Updated last week
- DWM (Domain Weight Modification) by Blackfrost_AI — self-instructing runbook package for Qwen 9B-class models and their fine-tunes☆15Oct 5, 2026Updated last month