NInfer on Tesla V100 (sm_70): Qwen3.8-27B deployment, build/convert scripts, systemd service, and cross-engine benchmark (vs llama.cpp). Fork of Neroued/ninfer with sm70 vision-encoding speedup.
☆53Sep 15, 2026Updated last month
Alternatives and similar repositories for ninfer-v100
Users that are interested in ninfer-v100 are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- 1Cat Studio: unified vLLM workbench, Codex Agent and creative canvas☆15Updated this week
- High-performance single-GPU inference for selected model checkpoints and GPUs.☆261Sep 14, 2026Updated last month
- vLLM fork for Tesla V100☆15May 31, 2026Updated 5 months ago
- llama.cpp optimized for NVIDIA Tesla V100 (SM70). Qwen3.8-Flash-Next (MoE) with MTP on 1/2/3/6x V100-16GB at 262K context (6 GPUs: prefil…☆73Updated this week
- Agentic-coding oriented inference engine for N x V100-SXM2, based on SGLang: GLM-5.3-Flash, Qwen3.8-Flash-Next, DeepSeek-V4.1-Flash, Mini…☆35Oct 7, 2026Updated last month
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- Auto-built Docker image of 1Cat-vLLM for NVIDIA Tesla V100 (sm_70/Volta) with CUDA 12.8 — rebuilt daily when upstream releases☆12Jul 22, 2026Updated 3 months ago
- V100 / SM70-focused vLLM engineering fork for modern LLM inference.☆1,480Updated this week
- Qwen3.8-27B in native NVFP4/FP8 on 2x PCIe Tesla V100-32GB (SM70): the PCIe runbook for v100-skinny + 1Cat-vLLM, with the 3 fixes that ma…☆16Sep 13, 2026Updated last month
- Stable, high-throughput inference server for multi-GPU NVIDIA V100 (SM70): Qwen3.8 Flash-Next with AWQ / NVFP4, multimodal input, CPU/dis…☆48Updated this week
- Hand-written NVFP4 W4A16 CUDA kernels for Volta☆181Aug 19, 2026Updated 2 months ago
- SGLang for cool dudes using V100☆83Updated this week
- The set-and-forget LLM engine for Pascal and Volta: PXQ codec + kernels, auto-tuned per card. Ready-to-run PXQ models in MODELS.md; bench…☆115Updated this week
- Configurations and scripts for working with the Tesla V100 32Gb.☆17Sep 8, 2026Updated 2 months ago
- Implementation of FlashAttention-2 for Nvidia Tesla V100 / Titan V☆217Updated this week
- 1-Click AI Models by DigitalOcean Gradient • AdDeploy popular AI models on DigitalOcean Gradient GPU virtual machines with just a single click. Zero configuration with optimized deployments.
- 两张 V100 跑 Qwen3.8-27B 到 100 tok/s（186K 上下文）：重写 sm70 注意力内核 + 多卡张量并行。Fork of geoffwatts/ninfer-v100.☆18Sep 28, 2026Updated last month
- Sol-Attn (arXiv 2607.24027) sparse acceleration for MiniMax H3 on V100 — single-node ComfyUI plugin (embedded FP16Safe + keep-or-drop ker…☆33Aug 29, 2026Updated 2 months ago
- This project is specifically developed for V100, based on lmdeploy 0.12.1, and supports mainstream open-source models from Q4 2025 to Q1 …☆22Mar 18, 2026Updated 7 months ago
- marlin_v100 是一个从 vLLM 主树中提取出来的最小 Marlin 独立开发工作区，聚焦于 Marlin dense 与 Marlin MoE 的源码开发、最小构建和轻量验证。它保留了核心 CUDA/C++ 实现、最小 Python 薄封装、生成器测试与主树回写…☆26Jul 2, 2026Updated 4 months ago
- A ComfyUI custom node enabling **Flash Attention 1** on legacy NVIDIA GPUs (Tesla V100, T4) that lack Compute Capability 8.0+ required by…☆33Feb 9, 2026Updated 8 months ago
- 1Cat-vLLM on Intel Gaudi☆47Oct 12, 2026Updated 3 weeks ago
- CUDA engine that splits every layer of a production quality MoE LLM across three PCIe V100 cards delivering 382 tok/s on a used $1,800 bo…☆68Updated this week
- xllama.cpp — Optimized LLM Inference for NVidia Tesla V100-SXM2-32GB☆17Apr 29, 2026Updated 6 months ago
- Block-FP8 (W8A16) on NVIDIA V100 / Volta (sm_70) for vLLM — the FP8 path upstream drops on Volta, tuned for sparse MoE.☆15Jul 10, 2026Updated 3 months ago
- End-to-end encrypted email - Proton Mail • AdSpecial offer: 40% Off Yearly / 80% Off First Month. All Proton services are open source and independently audited for security.
- LLM inference in C/C++☆22Oct 25, 2026Updated 2 weeks ago
- Mixed-precision acceleration patch for MiniMax H3 inference on NVIDIA V100☆39Sep 6, 2026Updated 2 months ago
- forked from vllm-project/flash-attention☆74May 9, 2026Updated 5 months ago
- ☆16Oct 20, 2026Updated 2 weeks ago
- A formally-verified compiler IR☆10May 5, 2026Updated 6 months ago
- High-performance single-GPU inference for RTX 5090 Mobile.☆21Oct 4, 2026Updated last month
- vLLM fork for Tesla V100 (SM70) — extends 1CatAI's AWQ support and adds GGUF support☆26Jun 20, 2026Updated 4 months ago
- ☆24Jun 27, 2026Updated 4 months ago
- Third party commentaries for UniqueBible☆13Apr 8, 2021Updated 5 years ago
- AI Agents on DigitalOcean Gradient AI Platform • AdBuild production-ready AI agents using customizable tools or access multiple LLMs through a single endpoint. Create custom knowledge bases or connect external data.
- Ternary (2.125 bit/weight) Bonsai 2 27B on the NINFER engine — Ada sm_89, Linux. Derivative work of Neroued/ninfer; see README.☆61Oct 23, 2026Updated 2 weeks ago
- DeepSeek V4 Flash 284B on AMD Strix Halo (gfx1151) — up to 32 tok/s decode & ~250 tok/s prefill via ROCmFPX, DSpark & ROCm 7.2☆20Aug 28, 2026Updated 2 months ago
- Run MiniMax H3 at native fp16 on pre-bf16 GPUs (V100/Volta) — exact-math fix for the black-frame overflows, ~11× faster than the fp32 fal…☆58Aug 8, 2026Updated 3 months ago
- High-performance single-GPU inference for selected model checkpoints and GPUs.☆3,293Updated this week
- VMX Hypervisor Windows kernel (x64) Toolbox (基于 Intel VT-x (VMX) 和 AMD SVM 的 Windows x64 Hypervisor 工具箱。通过在操作系统下方插入一层轻量级 Type-2 Hyperviso…☆36Jun 28, 2026Updated 4 months ago
- One line of NInfer for the Blackwell, Ada Lovelace, Ampere, Turing and Volta NVIDIA GPUs, consolidated from the forks that carry it: GGUF…☆204Updated this week
- Voice typing for Windows. local & cloud ASR (Seed-ASR 2.0 )豆包语音输入法,FireRedASR2,Qwen audio/ASR,microsoft/mai-transcribe-2,MIMO ASR☆20Nov 2, 2026Updated last week