llama.cpp optimized for NVIDIA Tesla V100 (SM70). Qwen3.8-Flash-Next (MoE) with MTP on 1/2/3/6x V100-16GB at 262K context (6 GPUs: prefill 3402 tok/s at 200K; 2 GPUs on one socket: 2152 tok/s). Qwen3.8-27B with DFlash2 on 1-6 GPUs, up to 512K context. Multimodal, thinking, concurrent serving, built-in web UI.
☆73Nov 3, 2026Updated this week
Alternatives and similar repositories for llama.cpp-v100
Users that are interested in llama.cpp-v100 are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Block-FP8 (W8A16) on NVIDIA V100 / Volta (sm_70) for vLLM — the FP8 path upstream drops on Volta, tuned for sparse MoE.☆15Jul 10, 2026Updated 3 months ago
- Stable, high-throughput inference server for multi-GPU NVIDIA V100 (SM70): Qwen3.8 Flash-Next with AWQ / NVFP4, multimodal input, CPU/dis…☆48Updated this week
- NInfer on Tesla V100 (sm_70): Qwen3.8-27B deployment, build/convert scripts, systemd service, and cross-engine benchmark (vs llama.cpp). …☆53Sep 15, 2026Updated last month
- ComfyUI MiniMax H3 inference optimizations for NVIDIA Tesla V100 (SM70)☆31Sep 29, 2026Updated last month
- V100 / SM70-focused vLLM engineering fork for modern LLM inference.☆1,480Updated this week
- GPUs on demand by Runpod - Special Offer Available • AdRun AI, ML, and HPC workloads on powerful cloud GPUs—without limits or wasted spend. Deploy GPUs in under a minute and pay by the second.
- Agentic-coding oriented inference engine for N x V100-SXM2, based on SGLang: GLM-5.3-Flash, Qwen3.8-Flash-Next, DeepSeek-V4.1-Flash, Mini…☆35Oct 7, 2026Updated last month
- Sol-Attn (arXiv 2607.24027) sparse acceleration for MiniMax H3 on V100 — single-node ComfyUI plugin (embedded FP16Safe + keep-or-drop ker…☆33Aug 29, 2026Updated 2 months ago
- Implementation of FlashAttention-2 for Nvidia Tesla V100 / Titan V☆217Updated this week
- Qwen3.8-27B in native NVFP4/FP8 on 2x PCIe Tesla V100-32GB (SM70): the PCIe runbook for v100-skinny + 1Cat-vLLM, with the 3 fixes that ma…☆16Sep 13, 2026Updated last month
- Hand-written NVFP4 W4A16 CUDA kernels for Volta☆181Aug 19, 2026Updated 2 months ago
- A 198t/s and 353t/s qwen 3.8 27b inferencing engine for 2*v100 and 4*v100☆102Updated this week
- Run local LLMs on a Tesla V100-SXM2-16GB: prebuilt binaries, serve scripts, and setup for Windows and Linux (Qwen3 + Gemma 4, Claude Code…☆20Sep 1, 2026Updated 2 months ago
- forked from vllm-project/flash-attention☆74May 9, 2026Updated 5 months ago
- Third party commentaries for UniqueBible☆13Apr 8, 2021Updated 5 years ago
- 1-Click AI Models by DigitalOcean Gradient • AdDeploy popular AI models on DigitalOcean Gradient GPU virtual machines with just a single click. Zero configuration with optimized deployments.
- SGLang for cool dudes using V100☆83Updated this week
- 1Cat-vLLM on Intel Gaudi☆47Oct 12, 2026Updated 3 weeks ago
- marlin_v100 是一个从 vLLM 主树中提取出来的最小 Marlin 独立开发工作区，聚焦于 Marlin dense 与 Marlin MoE 的源码开发、最小构建和轻量验证。它保留了核心 CUDA/C++ 实现、最小 Python 薄封装、生成器测试与主树回写…☆26Jul 2, 2026Updated 4 months ago
- FlashAttention brought back to Tesla V100 — a deep llama.cpp fork: SM 7.0 D256 kernels, SplitKV3, q4_0 KV cache, DFlash2 speculative deco…☆19Nov 1, 2026Updated last week
- xllama.cpp — Optimized LLM Inference for NVidia Tesla V100-SXM2-32GB☆17Apr 29, 2026Updated 6 months ago
- Captures the desktop in a loop using Desktop Duplication API and saves them as bitmap images in the current Directory☆14Dec 27, 2017Updated 8 years ago
- The set-and-forget LLM engine for Pascal and Volta: PXQ codec + kernels, auto-tuned per card. Ready-to-run PXQ models in MODELS.md; bench…☆115Updated this week
- Single-card Intel Gaudi2 vLLM source snapshot: FP8 Qwen, non-greedy DFlash2/MTP, GDN kernels and host-side optimizations.☆18Updated this week
- Mechanical models and drawings of Xilinx VCU1525 board, heatsink and other hardware.☆10Dec 7, 2018Updated 7 years ago
- Managed Kubernetes at scale on DigitalOcean • AdDigitalOcean Kubernetes includes the control plane, bandwidth allowance, container registry, automatic updates, and more for free.
- Golang package for PCI Express data transfers☆13Apr 24, 2018Updated 8 years ago
- Step-by-step guide to cross-flash the AMD Radeon Pro V620 to a Radeon Pro W6800 for display output, home-lab compatibility, and fan contr…☆16Sep 16, 2026Updated last month
- Known-good vLLM recipe: Qwen3.8-27B-Int8 + MTP speculative decoding on NVIDIA CMP 170HX 64GB (Hynix) — 84 tok/s, 262k ctx☆13Oct 8, 2026Updated last month
- ☆27Oct 1, 2026Updated last month
- Mixture of Lora Experts☆11Apr 7, 2024Updated 2 years ago
- HIP kernels for AMD Strix Halo (gfx1151) and a vLLM plugin that runs them on stock upstream vLLM: up to 31x faster prefill and ~2x decode…☆21Updated this week
- SQRL Port of ethminer☆11Feb 1, 2021Updated 5 years ago
- Intel Arc LLM inference server (A770/B60): Qwen hybrids up to Qwen3.8-Flash-Next (512-expert MoE through an expert cache in VRAM and host…☆26Updated this week
- Qwen3.8-27B on one DGX Spark GB10: DFlash 2 for 65 tok/s C1 and 120 C4; EAGLE 3/1/4 for ~191 C8, ~330 C16 and ~535 C32; native 262K, pinn…☆13Oct 6, 2026Updated last month
- AI Agents on DigitalOcean Gradient AI Platform • AdBuild production-ready AI agents using customizable tools or access multiple LLMs through a single endpoint. Create custom knowledge bases or connect external data.
- an sata controller using smallest resource.☆17Feb 5, 2014Updated 12 years ago
- PiG - Raspberry Pi for GPU cards☆14May 9, 2023Updated 3 years ago
- ☆18Oct 25, 2026Updated 2 weeks ago
- NVIDIA Tesla V100 显卡驱动自动化管理套件☆33Oct 9, 2025Updated last year
- ☆11Feb 11, 2015Updated 11 years ago
- ☆14Oct 13, 2026Updated 3 weeks ago
- C++ code for the fan shim on raspberry pi☆11Nov 28, 2019Updated 6 years ago