NVFP4 KV cache for vLLM on SM120 (RTX PRO 6000) via FlashInfer FA2 explicit-SF-stride patch — ~1.5x fp8 pool at ~95-104% speed
☆20Jun 5, 2026Updated 4 months ago
Alternatives and similar repositories for vllm-nvfp4-kv-sm120
Users that are interested in vllm-nvfp4-kv-sm120 are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- ☆37Apr 26, 2026Updated 5 months ago
- Hikari's curated English measured notes on local LLM systems (serving, specdec, training, steering).☆14Jul 29, 2026Updated 2 months ago
- A high-throughput and memory-efficient inference and serving engine for LLMs☆42Updated this week
- Qwen3.8-27B NVFP4 (W4A16 shipped recipe) + MTP on NVIDIA DGX Spark GB10/SM121 — vLLM v0.27.2rc0, FP8 KV, 262K context, full reproducibili…☆27Aug 20, 2026Updated 2 months ago
- Validated GLM-5.3 Flash recipe for 2x NVIDIA RTX PRO 6000 Blackwell 96GB: 262K context, EXL3/TR3, adaptive MTP, tools, and vision.☆61Sep 20, 2026Updated last month
- Serverless GPU API endpoints on Runpod - Get Bonus Credits • AdSkip the infrastructure headaches. Auto-scaling, pay-as-you-go, no-ops approach lets you focus on innovating your application.
- DeepSeek-V4.1-Flash-1017 on NVIDIA DGX Sparks with TensorFold + EXL3: TP4 (4 Sparks, 93.6 tok/s, 1M context) and TP2 (2 Sparks, 73 tok/s)…☆144Updated this week
- Local benchmarking UI for LLMs and AI agents☆24Apr 13, 2026Updated 6 months ago
- Truthful scheduler-step and streaming visualization for local LLM inference☆35Sep 6, 2026Updated last month
- LLM inference decode throughput benchmark with Rich TUI dashboard. Measures token generation speed across concurrency levels and context …☆120Oct 13, 2026Updated last week
- ☆16Jun 28, 2026Updated 3 months ago
- ☆11Aug 19, 2026Updated 2 months ago
- Sample code to demonstrate how to implement a simple chat that works on .NET MAUI, Blazor and Blazor Hybrid with SignalR☆10Feb 14, 2023Updated 3 years ago
- ☆23Updated this week
- GLM-5.3-Flash EXL3 (320B MoE) on 2x NVIDIA DGX Spark — production serving kit, 1M context, 97%+ multi-session prefix caching, DFlash2 spe…☆63Updated this week
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- GLM-5.3-Flash (NVFP4) at TP4 across 4x DGX Spark via a switchless RoCE ring + DFlash2 — reproducible recipe☆107Oct 15, 2026Updated last week
- Realtime object detection on Android with Xamarin.Android and TensorFlow Lite (SSD MobileNet).☆11Sep 15, 2026Updated last month
- Qwen 3.8 Flash NVFP4 with PLE NVFP4 quantized to run on a single DGX Spark☆15Aug 28, 2026Updated last month
- Prompt Engineering'i öğrenebileceğiniz Erdoğan Eker ile birlikte oluşturduğumuz bir kitap.☆12Sep 9, 2024Updated 2 years ago
- A user-friendly editor tool has been developed to create or edit JSON configuration files:☆12Sep 9, 2025Updated last year
- Flash-Flash KDA: H100-optimized Flash Kimi Delta Attention kernels☆24Jul 24, 2026Updated 2 months ago
- Plug-and-play PyTorch transformer research framework. Swap attention, FFN, norm, positional, connection, projection, optimizer, scheduler…☆18Updated this week
- Three-DGX-Spark TP3 serving recipe for GLM-5.3 Flash☆19Oct 5, 2026Updated 2 weeks ago
- Fused MoE kernel☆23Aug 10, 2026Updated 2 months ago
- Wordpress hosting with auto-scaling - Free Trial Offer • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- Experimental runtime reclamation of GB10 display-reserved memory into ordinary Linux RAM, with guarded driver, local signing, and reboot …☆11Sep 27, 2026Updated 3 weeks ago
- DGX Spark EC fan speed override / rollback tool☆20Aug 23, 2026Updated 2 months ago
- Open-source macOS AI computer use and native GUI automation on Apple silicon. Jev + OmniParser CoreML + Apple Vision OCR. Bring your own …☆24Sep 18, 2026Updated last month
- Security capability evaluation of locally served models. Method, corpus, and per-model results.☆11Sep 22, 2026Updated last month
- LLMs take the path of least resistance. Pre-Reasoning hands any model a map of the problem first, so it unlocks the knowledge it already …☆19Sep 21, 2026Updated last month
- EXLR8: a DGX Spark tuned quantization format☆13Updated this week
- Docker images for LLM inference (SGLang + vLLM) on NVIDIA Blackwell GPUs (SM120, CUDA 13.2)☆93Updated this week
- repo behind the site☆10Sep 29, 2015Updated 11 years ago
- Validated vLLM recipe: Qwen3.8-27B-NVFP4 at full 262,144-token context on a single RTX 5090, GPU-only (no CPU offload).☆16Aug 20, 2026Updated 2 months ago
- Deploy open-source AI quickly and easily - Special Bonus Offer • AdRunpod Hub is built for open source. One-click deployment and autoscaling endpoints without provisioning your own infrastructure.
- PlayList Animation using MAUI.☆14Nov 1, 2022Updated 3 years ago
- Notification integration for SimpleX messenger☆23Updated this week
- SGLang for GLM-5.3-Flash on two or four RTX PRO 6000 Blackwell GPUs (SM120), with TP2 W4A16 and TP4 NVIDIA NVFP4 profiles.☆46Oct 11, 2026Updated last week
- Give the DeepSeek Harness (dsh) eyes with any text model + any vision model: a vision proxy for chat image attachments and an analyze_ima…☆12Sep 28, 2026Updated 3 weeks ago
- GLM-5.2-NVFP4-REAP-469B serving on SM120 (4× RTX PRO 6000 Blackwell) — one-command vLLM launch recipe, 250K context, DeepSeek Sparse Atte…☆155Jun 19, 2026Updated 4 months ago
- ☆29Sep 7, 2026Updated last month
- New results and AI-powered research on the square packing problem☆43Updated this week