High-performance single-GPU inference for selected model checkpoints and GPUs.
☆255Sep 14, 2026Updated last month
Alternatives and similar repositories for ninfer-v100
Users that are interested in ninfer-v100 are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Strata port for Tesla V100 (sm_70): Qwen3.8-Flash-Next with 262K context and MTP☆102Updated this week
- Hand-written NVFP4 W4A16 CUDA kernels for Volta☆176Aug 19, 2026Updated 2 months ago
- 两张 V100 跑 Qwen3.8-27B 到 100 tok/s（186K 上下文）：重写 sm70 注意力内核 + 多卡张量并行。Fork of geoffwatts/ninfer-v100.☆16Sep 28, 2026Updated last month
- V100 / SM70-focused vLLM engineering fork for modern LLM inference.☆1,421Updated this week
- A 198t/s and 353t/s qwen 3.8 27b inferencing engine for 2*v100 and 4*v100☆81Updated this week
- Wordpress hosting with auto-scaling - Free Trial Offer • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- Run MiniMax H3 at native fp16 on pre-bf16 GPUs (V100/Volta) — exact-math fix for the black-frame overflows, ~11× faster than the fp32 fal…☆56Aug 8, 2026Updated 2 months ago
- Sol-Attn (arXiv 2607.24027) sparse acceleration for MiniMax H3 on V100 — single-node ComfyUI plugin (embedded FP16Safe + keep-or-drop ker…☆34Aug 29, 2026Updated 2 months ago
- A ComfyUI custom node enabling **Flash Attention 1** on legacy NVIDIA GPUs (Tesla V100, T4) that lack Compute Capability 8.0+ required by…☆33Feb 9, 2026Updated 8 months ago
- Stable, high-throughput inference server for multi-GPU NVIDIA V100 (SM70): Qwen3.8 Flash-Next with AWQ / NVFP4, multimodal input, CPU/dis…☆17Updated this week
- Flash Attention v2 for Pascal (P100, GTX 1080) and Volta (V100) GPUs — bringing O(N) memory attention to hardware the official flash-attn…☆27Apr 4, 2026Updated 6 months ago
- SGLang for cool dudes using V100☆82Oct 5, 2026Updated 3 weeks ago
- High-performance single-GPU inference for selected model checkpoints and GPUs.☆3,206Updated this week
- vLLM fork for Tesla V100☆15May 31, 2026Updated 4 months ago
- llama.cpp performance patches developed and measured on Tesla P100 (GP100, sm_60). Most are not Pascal-specific.☆56Updated this week
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- forked from vllm-project/flash-attention☆70May 9, 2026Updated 5 months ago
- Benchmarks and notes for running modern LLMs with vLLM on 8x Tesla V100-32GB in 2026.☆23Jul 10, 2026Updated 3 months ago
- The set-and-forget LLM engine for Pascal and Volta: PXQ codec + kernels, auto-tuned per card. Ready-to-run PXQ models in MODELS.md; bench…☆91Updated this week
- ENGRAFT: write facts into an LLM's n-gram (Engram) memory table as a removable overlay: 100 facts in a 9 MB file, no weights touched, ver…☆57Sep 30, 2026Updated 3 weeks ago
- marlin_v100 是一个从 vLLM 主树中提取出来的最小 Marlin 独立开发工作区，聚焦于 Marlin dense 与 Marlin MoE 的源码开发、最小构建和轻量验证。它保留了核心 CUDA/C++ 实现、最小 Python 薄封装、生成器测试与主树回写…☆26Jul 2, 2026Updated 3 months ago
- Optimized build of llama.cpp for gfx906/Vega20.☆19Oct 18, 2026Updated last week
- Qwen3.8-27B on a single RTX 4090 (sm_89): Cinference/NInfer port with MTP speculative decoding and K8V4 KV cache☆14Oct 20, 2026Updated last week
- Mixed-precision acceleration patch for MiniMax H3 inference on NVIDIA V100☆40Sep 6, 2026Updated last month
- Superseded by github.com/poisonxa16/pxa — PXA, the set-and-forget engine for Pascal and Volta☆24Sep 8, 2026Updated last month
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- Community-tested recipes, diagnostics, and benchmarks for CMP 170HX AI workloads☆90Oct 19, 2026Updated last week
- Qwen3.8-Flash-Next on any consumer hardware: one-click install for Windows / Linux. Strata inference engine, OpenAI/Anthropic API on loca…☆35,851Updated this week
- A guide for modding bios on Z440 / Z640 / Z840 HP workstations☆29Sep 21, 2026Updated last month
- Semantic Skill Space (SSS): Injecting skill embeddings into KV cache for small-model agents to recover skill behavior with lower prompt-c…☆15Mar 2, 2026Updated 7 months ago
- vLLM for NVIDIA CMP 170HX (Ampere sm_80): fast, repeatable model serving on one card or several☆18Oct 19, 2026Updated last week
- Project Maya - run GLM-5.3-Flash (321B MoE), DeepSeek-V4.1-Flash-1017 (748B MoE) and MiniMax-M3.1-Flash (428B MoE) privately on your own …☆2,190Updated this week
- ☆18Oct 22, 2026Updated last week
- Native DeepSeek V4.1 Flash inference across one RTX PRO 6000 and four DGX Spark expert workers☆38Sep 25, 2026Updated last month
- KVMem virtualizes KV memory across inference backends to make ultra-long-context local AI accessible on limited hardware.☆1,187Updated this week
- Virtual machines for every use case on DigitalOcean • AdGet dependable uptime with 99.99% SLA, simple security tools, and predictable monthly pricing with DigitalOcean's virtual machines, called Droplets.
- Portable optimized ROCm and Qwen3.8-27B setup for AMD Strix Halo☆86Sep 16, 2026Updated last month
- A scsi emulator based off of the work of http://www.geocities.jp/kugimoto0715/rascsi/index.html☆21Nov 19, 2017Updated 8 years ago
- A tool to unlobotomize your NVIDIA card!☆815Updated this week
- ☆492Oct 20, 2026Updated last week
- DeepSeek-V4-Flash-0731 on 4x CMP 170HX (sm_80): 98 tok/s decode, ~5300 tok/s prefill. DSpark speculative decoding under pipeline parallel…☆118Aug 13, 2026Updated 2 months ago
- Proxima lets existing GPUs serve 4x more concurrent requests☆26Aug 19, 2026Updated 2 months ago
- A tool to unlobotomize your NVIDIA card!☆31Sep 6, 2026Updated last month