High-performance single-GPU inference for selected model checkpoints and GPUs.
☆274Sep 14, 2026Updated 2 months ago
Alternatives and similar repositories for ninfer-v100
Users that are interested in ninfer-v100 are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Strata port for Tesla V100 (sm_70): Qwen3.8-Flash-Next with 262K context and MTP☆157Updated this week
- High-performance Duo-V100-GPU inference for selected model checkpoints and GPUs.☆42Nov 2, 2026Updated 2 weeks ago
- V100 / SM70-focused vLLM engineering fork for modern LLM inference.☆1,533Updated this week
- A 198t/s and 353t/s qwen 3.8 27b inferencing engine for 2*v100 and 4*v100☆119Updated this week
- Hand-written NVFP4 W4A16 CUDA kernels for Volta☆188Aug 19, 2026Updated 2 months ago
- Managed Database hosting by DigitalOcean • AdPostgreSQL, MySQL, MongoDB, Kafka, Valkey, and OpenSearch available. Automatically scale up storage and focus on building your apps.
- High-performance single-GPU inference for selected model checkpoints and GPUs.☆16Sep 26, 2026Updated last month
- SGLang for cool dudes using V100☆86Nov 11, 2026Updated last week
- Sol-Attn (arXiv 2607.24027) sparse acceleration for MiniMax H3 on V100 — single-node ComfyUI plugin (embedded FP16Safe + keep-or-drop ker…☆35Aug 29, 2026Updated 2 months ago
- Run MiniMax H3 at native fp16 on pre-bf16 GPUs (V100/Volta) — exact-math fix for the black-frame overflows, ~11× faster than the fp32 fal…☆62Aug 8, 2026Updated 3 months ago
- One line of NInfer for the Blackwell, Ada Lovelace, Ampere, Turing and Volta NVIDIA GPUs, consolidated from the forks that carry it: GGUF…☆232Updated this week
- The set-and-forget LLM engine for Pascal and Volta: PXQ codec + kernels, auto-tuned per card. Ready-to-run PXQ models in MODELS.md; bench…☆126Updated this week
- marlin_v100 是一个从 vLLM 主树中提取出来的最小 Marlin 独立开发工作区,聚焦于 Marlin dense 与 Marlin MoE 的源码开发、最小构建和轻量验证。它保留了核心 CUDA/C++ 实现、最小 Python 薄封装、生成器测试与主树回写…☆27Jul 2, 2026Updated 4 months ago
- Fork of llama.cpp Nvida Volta V100 (tensor parallelism 4 x v100 GPU)☆12Updated this week
- forked from vllm-project/flash-attention☆75May 9, 2026Updated 6 months ago
- AI Agents on DigitalOcean Gradient AI Platform • AdBuild production-ready AI agents using customizable tools or access multiple LLMs through a single endpoint. Create custom knowledge bases or connect external data.
- High-performance single-GPU inference for selected model checkpoints and GPUs.☆3,379Updated this week
- A ComfyUI custom node enabling **Flash Attention 1** on legacy NVIDIA GPUs (Tesla V100, T4) that lack Compute Capability 8.0+ required by…☆35Feb 9, 2026Updated 9 months ago
- Flash Attention v2 for Pascal (P100, GTX 1080) and Volta (V100) GPUs — bringing O(N) memory attention to hardware the official flash-attn…☆32Nov 3, 2026Updated 2 weeks ago
- Mixed-precision acceleration patch for MiniMax H3 inference on NVIDIA V100☆40Sep 6, 2026Updated 2 months ago
- Optimized build of llama.cpp for gfx906/Vega20.☆26Nov 10, 2026Updated last week
- ComfyUI custom node: run MiniMax H3 at near-fp16 speed with near-fp32 numerical stability on GPUs without bf16/fp8 hardware (V100 sm_70)☆22Nov 10, 2026Updated last week
- ENGRAFT: write facts into an LLM's n-gram (Engram) memory table as a removable overlay: 100 facts in a 9 MB file, no weights touched, ver…☆62Sep 30, 2026Updated last month
- FlashAttention brought back to Tesla V100 — a deep llama.cpp fork: SM 7.0 D256 kernels, SplitKV3, q4_0 KV cache, DFlash2 speculative deco…☆19Nov 8, 2026Updated last week
- llama.cpp fork for two V100 32GB cards to run Qwen3.8FlashNext☆15Nov 3, 2026Updated 2 weeks ago
- 1-Click AI Models by DigitalOcean Gradient • AdDeploy popular AI models on DigitalOcean Gradient GPU virtual machines with just a single click. Zero configuration with optimized deployments.
- Run GLM-5.3 (744B), GLM-5.3-Flash (321B), DeepSeek-V4.1-Flash (748B) and MiniMax-M3.1-Flash (428B) on your own GPUs - one engine, one das…☆3,018Updated this week
- Benchmarks and notes for running modern LLMs with vLLM on 8x Tesla V100-32GB in 2026.☆25Jul 10, 2026Updated 4 months ago
- CUDA engine that splits every layer of a production quality MoE LLM across three PCIe V100 cards delivering 382 tok/s on a used $1,800 bo…☆84Nov 7, 2026Updated last week
- Strata 0.2.3 (engine 0.1.48) for AMD gfx906 (MI50/MI60/Radeon VII): rocBLAS solution table, Qwen3.6/Ornith engine restored, DFlash2 build…☆28Nov 1, 2026Updated 2 weeks ago
- A cmpunlocker fork with working p2p on SYS-4028GR-TRT2☆27Aug 8, 2026Updated 3 months ago
- A tool to unlobotomize your NVIDIA card!☆925Oct 28, 2026Updated 3 weeks ago
- ML software (llama.cpp, ComfyUI, Strata, Maya, ...) builds for AMD gfx906 GPUs, e.g. Radeon VII / MI50 / MI60☆387Updated this week
- Qwen3.8-Flash-Next on any consumer hardware: one-click install for Windows / Linux. Strata inference engine, OpenAI/Anthropic API on loca…☆45,204Updated this week
- Superseded by github.com/poisonxa16/pxa — PXA, the set-and-forget engine for Pascal and Volta☆25Sep 8, 2026Updated 2 months ago
- Virtual machines for every use case on DigitalOcean • AdGet dependable uptime with 99.99% SLA, simple security tools, and predictable monthly pricing with DigitalOcean's virtual machines, called Droplets.
- KVMem virtualizes KV memory across inference backends to make ultra-long-context local AI accessible on limited hardware.☆1,357Updated this week
- Qwen3.8-27B on a single RTX 4090 (sm_89): Cinference/NInfer port with MTP speculative decoding and K8V4 KV cache☆15Nov 1, 2026Updated 2 weeks ago
- Serve large Qwen models fast on the GPUs you actually own. Qwen3.8-27B on a single 24 GB card with vLLM: 127 tok/s single-user (381 when …☆2,751Nov 10, 2026Updated last week
- A LD_PRELOAD library to share CUDA VRAM weights across multiple processes using IPC.☆12Jun 30, 2026Updated 4 months ago
- vLLM fork for Tesla V100 (SM70) — extends 1CatAI's AWQ support and adds GGUF support☆27Jun 20, 2026Updated 4 months ago
- The main repository for building Pascal-compatible versions of ML applications and libraries.☆220Aug 23, 2025Updated last year
- Community-tested recipes, diagnostics, and benchmarks for CMP 170HX AI workloads☆101Nov 9, 2026Updated last week
Are these results useful?