RTX 6000 Pro Wiki — Running Large LLMs (Qwen3.5-397B, Kimi-K2.5, GLM-5) on PCIe GPUs without NVLink
☆1,162Oct 30, 2026Updated this week
Alternatives and similar repositories for rtx6kpro
Users that are interested in rtx6kpro are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Docker images for LLM inference (SGLang + vLLM) on NVIDIA Blackwell GPUs (SM120, CUDA 13.2)☆96Oct 20, 2026Updated last week
- Archived: b12x development has moved to FlashInfer. See README for migration and PR-porting guidance.☆268Oct 6, 2026Updated 3 weeks ago
- LLM inference decode throughput benchmark with Rich TUI dashboard. Measures token generation speed across concurrency levels and context …☆121Updated this week
- Optimized SGLang runtime for Qwen3.8-27B FP8 with DFlash2 and Qwen3.8 Flash-Next NVFP4 with FR-Spec on one NVIDIA RTX PRO 6000 Blackwell …☆187Updated this week
- A vLLM patch + hand‑written SM120 SASS kernels: 2‑bit MoE experts + an FP4 "delta" cache that recovers precision — matching the official …☆561Updated this week
- AI Agents on DigitalOcean Gradient AI Platform • AdBuild production-ready AI agents using customizable tools or access multiple LLMs through a single endpoint. Create custom knowledge bases or connect external data.
- A high-throughput and memory-efficient inference and serving engine for LLMs☆44Updated this week
- Validated GLM-5.3 Flash recipe for 2x NVIDIA RTX PRO 6000 Blackwell 96GB: 262K context, EXL3/TR3, adaptive MTP, tools, and vision.☆61Sep 20, 2026Updated last month
- Reproducible SGLang image for DeepSeek-V4-Flash-0731 on SM120☆37Aug 25, 2026Updated 2 months ago
- SGLang for GLM-5.3-Flash on two or four RTX PRO 6000 Blackwell GPUs (SM120), with TP2 W4A16 and TP4 NVIDIA NVFP4 profiles.☆47Oct 11, 2026Updated 2 weeks ago
- ☆704Updated this week
- DeepSeek V4.1 Flash on 4x RTX PRO 6000 Blackwell: native weights, DSpark, vision, NVMe or locked RAM Engram offload.☆84Oct 17, 2026Updated 2 weeks ago
- Qwen3.8-Flash-Next 180B NVFP4 @ 231 tok/s on one RTX PRO 6000 Blackwell (sm120) - sglang patches, launchers, full optimization analysis☆30Aug 31, 2026Updated 2 months ago
- Qwen3.8-27B on SGLang for NVIDIA RTX PRO 6000 (96GB, SM120): DSpark speculative decoding, full 256K context, 8 concurrent requests☆60Aug 24, 2026Updated 2 months ago
- RTX 6000 Pro Wiki — Running Large LLMs (Qwen3.5-397B, Kimi-K2.5, GLM-5) on PCIe GPUs without NVLink☆18Oct 22, 2026Updated last week
- Managed Kubernetes at scale on DigitalOcean • AdDigitalOcean Kubernetes includes the control plane, bandwidth allowance, container registry, automatic updates, and more for free.
- GLM-5.3-Flash EXL3 on 4x RTX PRO 6000 with TensorFold: one-command recipe and engine patches. By Aevonix Research in collaboration with M…☆10Oct 8, 2026Updated 3 weeks ago
- ☆37Apr 26, 2026Updated 6 months ago
- DeepSeek-v4-Flash 0731 recipe for 2x DGX Sparks☆1,504Oct 4, 2026Updated 3 weeks ago
- Educational reference: NVIDIA Blackwell SM100 vs SM120, NVFP4, tcgen05, MoE inference on consumer Blackwell☆29Sep 3, 2026Updated last month
- RTX 6000 96GB/Spark 128GB vLLM Recipe for Qwen 3.8 Flash Next☆23Sep 8, 2026Updated last month
- Community recipes for serving LLMs on RTX 3090/4090/5090 CUDA gpus. Multi-engine (vLLM, sglang, llama.cpp and other custom engines) and m…☆2,466Updated this week
- GLM 5.3 Flash for 2x RTX 6000, Full Mixed EXL3 Support, Code C1 >220tps☆22Oct 25, 2026Updated last week
- LLM speculative inference server for heterogeneous hardware & consumer GPUs☆2,967Updated this week
- ☆17Jun 17, 2026Updated 4 months ago
- Wordpress hosting with auto-scaling - Free Trial Offer • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- High-performance single-GPU inference for selected model checkpoints and GPUs.☆3,223Updated this week
- SGLang 0.5.21 for Qwen3.8-Flash-Next on one RTX PRO 6000: 264.5 tok/s decode, W4A16 routed experts (KLD 0.0111 vs BF16), 262K context, ex…☆17Updated this week
- Truthful scheduler-step and streaming visualization for local LLM inference☆35Sep 6, 2026Updated last month
- vLLM inference solutions for NVIDIA GB10 (DGX Spark, sm121) — KIMI-K3 B12X_MLA + DSpark, GLM-5.2☆67Sep 30, 2026Updated last month
- GLM-5.2-NVFP4-REAP-469B serving on SM120 (4× RTX PRO 6000 Blackwell) — one-command vLLM launch recipe, 250K context, DeepSeek Sparse Atte…☆155Jun 19, 2026Updated 4 months ago
- An optimized quantization and inference library for running LLMs locally on modern consumer-class GPUs☆1,772Updated this week
- Unlock P2P comms between consumer NVIDIA GPUs☆684Updated this week
- llama.cpp fork with additional SOTA quants and improved performance☆3,354Updated this week
- GLM-5.2 UD-Q2 decode speedup: CPU∥GPU MoE expert-split (harness, specs, de-risk)☆23Aug 9, 2026Updated 2 months ago
- Managed Database hosting by DigitalOcean • AdPostgreSQL, MySQL, MongoDB, Kafka, Valkey, and OpenSearch available. Automatically scale up storage and focus on building your apps.
- Run DeepSeek-V4.1-Flash / DeepSeek-V4-Flash and GLM-5.3-Flash on SM89 (Ada / RTX 4090) and SM120 (RTX PRO 6000) with vLLM☆198Updated this week
- Docker configuration for running VLLM on dual DGX Sparks☆2,463Updated this week
- Open-source AI computer designs, from desktop workstations to 8-GPU servers.☆1,548Oct 6, 2026Updated 3 weeks ago
- Messy repo filled with messy tests about hardware and LLMs. Built for me, public for you.☆47Sep 27, 2026Updated last month
- TokenSpeed is a speed-of-light LLM inference engine.☆2,830Updated this week
- Docker compose serving stack for DeepSeek v4 Flash DSpark for NVIDIA Spark GB10 system using Aidendle94 image☆33Aug 30, 2026Updated 2 months ago
- Control panel for VLLM, Sglang, llama.cpp, exllamav3☆1,836Updated this week