The definitive vLLM runtime for dual RTX 2080 Ti 22GB + NVLink, delivering Qwen 27B local inference with maximum 200+ tok/s single-request decode with support of FP8 weight ( Join Discord //discord.gg/VFqVVySdMS )
☆1,215Oct 1, 2026Updated last month
Alternatives and similar repositories for vLLM-2080Ti-Definitive
Users that are interested in vLLM-2080Ti-Definitive are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Single-request LLM serving recipes, patches, and benchmarks for modified RTX 2080 Ti 22GB / SM75 systems☆99Jun 10, 2026Updated 5 months ago
- ☆86Oct 31, 2026Updated last week
- 这是一套 MiniMax H3 本地部署与性能优化整理：在 4 张 22 GB RTX 2080 Ti 上，通过 Raylight FSDP/USP、双组NVLink、多卡 Qwen 文本编码节点、SM75 SageAttention 和通信流水，让MiniMax H3 的…☆27Aug 12, 2026Updated 3 months ago
- Reproducible vLLM deployment for Qwen3.8-27B on dual RTX 2080 Ti 22GB with NVLink☆22Sep 7, 2026Updated 2 months ago
- Local LLM Inference Speed Test Tool☆252Oct 21, 2026Updated 3 weeks ago
- AI Agents on DigitalOcean Gradient AI Platform • AdBuild production-ready AI agents using customizable tools or access multiple LLMs through a single endpoint. Create custom knowledge bases or connect external data.
- Strata v0.1.42 tuning for 2x RTX 2080 Ti 22G (Qwen3.8-Flash-Next Swift IQ3_XXS): 60K/128K prefill 2116/2163 tok/s on 48GB RAM☆15Oct 14, 2026Updated 3 weeks ago
- fastllm是后端无依赖的高性能大模型推理库，帮助您在大部分设备上高效推理大模型！☆5,130Updated this week
- Qwen3.8-Flash-Next on any consumer hardware: one-click install for Windows / Linux. Strata inference engine, OpenAI/Anthropic API on loca…☆43,454Updated this week
- LvLLM is a special NUMA extension of vllm that makes full use of CPU and memory resources, reduces GPU memory requirements, and features …☆474Nov 6, 2026Updated last week
- ☆64Sep 26, 2026Updated last month
- Vendor-neutral collective communication for GPU groups without a common native collective library, plus a PCIe/BAR1 direct path for consu…☆13Aug 30, 2026Updated 2 months ago
- ☆150Jun 13, 2026Updated 4 months ago
- 2080ti TU102 doing the unlock trick☆148Oct 22, 2026Updated 3 weeks ago
- Run MoE models beyond VRAM on AMD, NVIDIA & Intel GPUs. Vulkan-powered, portable Windows app with automatic setup. | 让游戏显卡跑起超显存大模型，支持 AMD…☆357Updated this week
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- High-performance single-GPU inference for selected model checkpoints and GPUs.☆3,332Updated this week
- Sage attention for turning.☆86Dec 29, 2025Updated 10 months ago
- MiniMax H3 video generation on 2080Ti 22G (Turing sm_75) - field-tested handbook, workflows & FAQ☆24Updated this week
- Inference engine for large language models on consumer GPUs, with deep optimization for older hardware☆55Oct 20, 2026Updated 3 weeks ago
- 1Cat Studio: unified vLLM workbench, Codex Agent and creative canvas☆15Updated this week
- Experimental NVIDIA tensor-compute unlock research: guarded TU102 initramfs port, offline tests, and Ampere reproduction notes☆403Nov 1, 2026Updated last week
- A reproducible local LLM benchmarking platform with versioned datasets, deterministic scoring, and auditable reports.☆181Oct 28, 2026Updated 2 weeks ago
- Flash Attention 2 implementation for Turing GPUs☆134Sep 14, 2026Updated last month
- A tool to unlobotomize your NVIDIA card!☆841Oct 28, 2026Updated 2 weeks ago
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- SGLang for cool dudes using V100☆86Updated this week
- 双 SM80 / CMP 170HX 的 Qwen3.8-Flash-Next 社区 vLLM：NVIDIA NVFP4、GDS、TEP2、MTP6，支持 1～10 并发，词表与 PCIe 通信优化。中英文部署与可构建源码。☆28Oct 15, 2026Updated 3 weeks ago
- Qwen3.8-Flash-Next (W4A16) on 2x CMP 170HX: 110 tok/s decode, 7K prefill, 800 tok/s @ 23 concurrent - runs on 38 GB home RAM, no pinned m…☆20Sep 18, 2026Updated last month
- A fork of jasl/vllm (codex/ds4-sm120-min-enable) with CPU-GPU hybrid inference support for DeepSeek-V4 on SM120+.☆32Sep 10, 2026Updated 2 months ago
- Async PLE SSD offload for Qwen3.8 Flash Next on a 64 GiB SM80 GPU: vLLM patch, setup guide, and measured PP/TG results.☆23Sep 24, 2026Updated last month
- 解锁自有 NVIDIA CMP 170HX / 90HX 的算力限制。Linux x86-64，需要 root。☆62Aug 6, 2026Updated 3 months ago
- LlamaServer大语言模型引擎的启动器，配置集中管理..☆13Aug 7, 2026Updated 3 months ago
- forked from vllm-project/flash-attention☆74May 9, 2026Updated 6 months ago
- DeepSeek V4 Flash DSpark DGX Spark TP4 Ring 生产调优与算子工程归档 (LuZ0.4.5)☆17Sep 11, 2026Updated 2 months ago
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- Lsglang is a special extension of sglang that fully utilizes CPU and GPU computing resources with an efficient GPU parallel + NUMA parall…☆168Updated this week
- Run GLM-5.3 (744B), GLM-5.3-Flash (321B), DeepSeek-V4.1-Flash (748B) and MiniMax-M3.1-Flash (428B) on your own GPUs - one engine, one das…☆2,827Updated this week
- High-performance Duo-V100-GPU inference for selected model checkpoints and GPUs.☆40Nov 2, 2026Updated last week
- vLLM fork for Tesla V100☆15May 31, 2026Updated 5 months ago
- Sol-Attn (arXiv 2607.24027) sparse acceleration for MiniMax H3 on V100 — single-node ComfyUI plugin (embedded FP16Safe + keep-or-drop ker…☆34Aug 29, 2026Updated 2 months ago
- Hand-written NVFP4 W4A16 CUDA kernels for Volta☆185Aug 19, 2026Updated 2 months ago
- The most comprehensive community resource for the NVIDIA CMP 170HX.☆134Jul 30, 2026Updated 3 months ago
Are these results useful?