One line of NInfer for the Blackwell, Ada Lovelace, Ampere, Turing and Volta NVIDIA GPUs, consolidated from the forks that carry it: GGUF block formats, ternary Bonsai 2, device route profiles, MTP and DFlash2.
☆196Nov 6, 2026Updated this week
Alternatives and similar repositories for ninfer-all
Users that are interested in ninfer-all are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- NInfer for Windows and around 16GB VRAM: RTX 5070 Ti / 5080 / 5090, Qwen3.8-27B GSQ-RCO Q3, CUDA 13 Native engine, tray manager, model co…☆64Oct 1, 2026Updated last month
- High-performance single-GPU inference for selected model checkpoints and GPUs.☆3,269Updated this week
- ☆16Oct 20, 2026Updated 2 weeks ago
- Run large AI models on your own PC: Qwen3.8-Flash-Next on Strata's engine, and GLM 5.3 Flash, DeepSeek V4 Flash, LFM2.5, Qwen3.6, Ornith …☆887Updated this week
- ☆94Updated this week
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- High-performance single-GPU inference for RTX 5090 Mobile.☆21Oct 4, 2026Updated last month
- ☆50Updated this week
- Ternary (2.125 bit/weight) Bonsai 2 27B on the NINFER engine — Ada sm_89, Linux. Derivative work of Neroued/ninfer; see README.☆61Oct 23, 2026Updated 2 weeks ago
- Qwen3.8-Flash-Next (125B MoE) up to 1M context on one or two RTX cards (tuned on 3080 10gb). A measured, tuned and benchmarked fork of …☆43Updated this week
- Hyper optimised Qwen3.8-27B inference on one RTX 3090: ReplaySSM, MTP3, reasoning effort, C1-C8 batching, and native Windows and Linux bu…☆502Updated this week
- High-performance single-GPU inference for selected model checkpoints and GPUs.☆260Sep 14, 2026Updated last month
- DeepSeek Harness QoL plugin for the local Qwen3.8 line (27B/Flash-Next): per-request thinking budgets, a compaction backend that stops bu…☆25Updated this week
- ☆11Oct 18, 2026Updated 2 weeks ago
- Xolotl (formerly Strata-DSV4.1): run DeepSeek-V4.1-Flash, all 552B, as a local coding agent on one gaming PC (RTX 4090 + 90 GB RAM) at up…☆467Updated this week
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- SDNQ support for ComfyUI☆20Jan 6, 2026Updated 10 months ago
- Qwen3.8-27B on one RTX 4090: full native 262K context via E8 4-bit KV, up to 149 tok/s code decode with MTP3, sm_89-retuned attention pre…☆15Sep 29, 2026Updated last month
- ☆529Oct 20, 2026Updated 2 weeks ago
- LLM inference in C/C++☆15Oct 30, 2026Updated last week
- Qwen3.8-27B on a single RTX 4090 (sm_89): Cinference/NInfer port with MTP speculative decoding and K8V4 KV cache☆14Updated this week
- high-performance linear attention kernel library built on TileLang☆15Jun 8, 2026Updated 4 months ago
- Quasar is an inference engine for Qwen3.6-27B-MTP / Qwen3.8-27B on consumer NVIDIA cards as well as Apple Silicon, in the spirit of antir…☆51Updated this week
- Qwen3.8-Flash-Next IQ3_S on 12 GB VRAM + 32 GB RAM + NVMe: ~45-50 tok/s decode, 131k context, Strata unify fork☆19Oct 24, 2026Updated last week
- Run Qwen3.8-27B at 140 tok/s on a single RTX 3090, up to 2x llama.cpp. Fast local LLM inference server with OpenAI- and Anthropic-compati…☆72Updated this week
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- Unofficial ComfyUI implementation of UniWorld.☆22Jun 10, 2025Updated last year
- NInfer on Tesla V100 (sm_70): Qwen3.8-27B deployment, build/convert scripts, systemd service, and cross-engine benchmark (vs llama.cpp). …☆53Sep 15, 2026Updated last month
- Qwen3.8-27B on one RTX 4090: full native 262K context via E8 4-bit KV, up to 149 tok/s code decode with MTP3, sm_89-retuned attention pre…☆228Updated this week
- Safely run your agents using any coding harness☆83Updated this week
- Serve large Qwen models fast on the GPUs you actually own. Qwen3.8-27B on a single 24 GB card with vLLM: 127 tok/s single-user (381 when …☆2,580Updated this week
- A full, unpruned 177B MoE model at 11.5 tok/s on one 12 GB GPU. Expert streaming for llama.cpp, with the measurement harness and every de…☆25Oct 3, 2026Updated last month
- Run GLM-5.3-Flash (321B), DeepSeek-V4.1-Flash (748B) and MiniMax-M3.1-Flash (428B) on your own GPUs - one engine, one dashboard, OpenAI- …☆2,585Updated this week
- LLM inference in C/C++☆471Updated this week
- Reference-driven Web 3D modeling skill for AI coding assistants: geometry, materials, lighting, procedural assets, and dynamic video veri…☆18Sep 6, 2026Updated 2 months ago
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- Strata fork: Qwen3.8-Flash-Next 125B MoE in NVFP4 on one RTX 20-50 card (12 GB+) and 64 GB of RAM or more. Setup installs our GPTQ quants…☆224Oct 24, 2026Updated last week
- Qwen3.8-Flash-Next on any consumer hardware: one-click install for Windows / Linux. Strata inference engine, OpenAI/Anthropic API on loca…☆40,925Updated this week
- Desktop UI for the NInfer local LLM inference engine☆33Updated this week
- More MoEs: a fork of Strata (Qwen3.8-Flash-Next, Qwen3.6/Ornith and more on one GPU + system RAM)☆25Updated this week
- ninfer-ext: Forked for speed.☆26Oct 22, 2026Updated 2 weeks ago
- Qwen3.8-27B in native NVFP4/FP8 on 2x PCIe Tesla V100-32GB (SM70): the PCIe runbook for v100-skinny + 1Cat-vLLM, with the 3 fixes that ma…☆16Sep 13, 2026Updated last month
- Source-available C++/CUDA inference engine for Qwen3.8-27B INT4 on 3x RTX 3090☆41Updated this week