High-performance single-GPU inference for selected model checkpoints and GPUs.
☆3,379Nov 17, 2026Updated this week
Alternatives and similar repositories for ninfer
Users that are interested in ninfer are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- High-performance single-GPU inference for selected model checkpoints and GPUs.☆185Sep 30, 2026Updated last month
- Hyper optimised Qwen3.8-27B inference on one RTX 3090: ReplaySSM, MTP3, reasoning effort, C1-C8 batching, and native Windows and Linux bu…☆523Oct 31, 2026Updated 2 weeks ago
- Qwen3.8-Flash-Next on any consumer hardware: one-click install for Windows / Linux. Strata inference engine, OpenAI/Anthropic API on loca…☆45,204Updated this week
- KVMem virtualizes KV memory across inference backends to make ultra-long-context local AI accessible on limited hardware.☆1,357Updated this week
- Serve large Qwen models fast on the GPUs you actually own. Qwen3.8-27B on a single 24 GB card with vLLM: 127 tok/s single-user (381 when …☆2,751Nov 10, 2026Updated last week
- Managed Kubernetes at scale on DigitalOcean • AdDigitalOcean Kubernetes includes the control plane, bandwidth allowance, container registry, automatic updates, and more for free.
- C++/CUDA single-GPU inference engine for Qwen3.8 (incl. Qwen3.8-Flash-Next with offloaded experts) on the RTX 5090; grew from NInfer☆303Updated this week
- Native Windows port of NInfer engine for RTX 5090. Features Qwen3.8-27B with QUASAR and NInfer models, MTP/DFlash2 with vision and 262,14…☆94Updated this week
- Local inference engine optimized for NVIDIA Blackwell☆1,259Updated this week
- One line of NInfer for the Blackwell, Ada Lovelace, Ampere, Turing and Volta NVIDIA GPUs, consolidated from the forks that carry it: GGUF…☆232Updated this week
- Qwen3.8-27B on one RTX 4090: full native 262K context via E8 4-bit KV, up to 149 tok/s code decode with MTP3, sm_89-retuned attention pre…☆239Oct 31, 2026Updated 2 weeks ago
- Run GLM-5.3 (744B), GLM-5.3-Flash (321B), DeepSeek-V4.1-Flash (748B) and MiniMax-M3.1-Flash (428B) on your own GPUs - one engine, one das…☆3,018Updated this week
- Strata fork: Qwen3.8-Flash-Next 125B MoE in NVFP4 on one RTX 20-50 card (12 GB+) and 64 GB of RAM or more. Setup installs our GPTQ quants…☆249Updated this week
- FreeToken brings datacenter-scale model serving to your desktop. Run massive models locally, fast and efficiently.☆17,289Updated this week
- NInfer port for Qwen 3.8 27B on RTX 4090☆187Updated this week
- Proton VPN Special Offer - Get 70% off • AdSpecial partner offer. Trusted by over 100 million users worldwide. Tested, Approved and Recommended by Experts.
- Lucebox inference engine for heterogeneous hardware & consumer GPUs☆3,016Updated this week
- ☆593Oct 20, 2026Updated 3 weeks ago
- High-performance single-GPU inference for selected model checkpoints and GPUs.☆25Oct 13, 2026Updated last month
- llama.cpp fork with additional SOTA quants and improved performance☆3,403Updated this week
- High-performance single-GPU inference for selected model checkpoints and GPUs.☆274Sep 14, 2026Updated 2 months ago
- An optimized quantization and inference library for running LLMs locally on modern consumer-class GPUs☆1,840Updated this week
- ☆131Updated this week
- V100 / SM70-focused vLLM engineering fork for modern LLM inference.☆1,533Updated this week
- Community recipes for serving LLMs on RTX 3090/4090/5090 CUDA gpus. Multi-engine (vLLM, sglang, llama.cpp and other custom engines) and m…☆2,505Updated this week
- Wordpress hosting with auto-scaling - Free Trial Offer • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- LLM Inference Engine for Metal, CUDA and Vulkan.☆2,322Updated this week
- KVarN, KV cache precision tail, low-bit quants in llama.cpp for longer context of better precision in the same VRAM☆1,197Updated this week
- ninfer-ext: Forked for speed.☆26Oct 22, 2026Updated 3 weeks ago
- Run large AI models on your own PC: Qwen3.8-Flash-Next on Strata's engine, and GLM 5.3 Flash, DeepSeek V4 Flash, LFM2.5, Qwen3.6, Ornith …☆1,163Updated this week
- Unlock P2P comms between consumer NVIDIA GPUs☆735Oct 26, 2026Updated 3 weeks ago
- The fastest inference engine for Qwen3.8-27B on the NVIDIA RTX 5090: up to 500 tokens/s for one agent and up to 2,000 tokens/s for many, …☆85Nov 4, 2026Updated 2 weeks ago
- Reliable model swapping for any local AI server. Chosen by thousands of users. Batteries included. If we don't have it, you probably don…☆6,263Updated this week
- An all-in-one, pure C++ inference engine for audio models, powered by ggml. Supports TTS, STT, VAD, voice conversion, music generation, a…☆4,318Updated this week
- Your GeForce runs AI at half speed. This fixes it for 20/30/40 series cards. Up to 2× FP16 / BF16 / TF32 tensor throughput☆20,359Updated this week
- AI Agents on DigitalOcean Gradient AI Platform • AdBuild production-ready AI agents using customizable tools or access multiple LLMs through a single endpoint. Create custom knowledge bases or connect external data.
- LLM inference in C/C++☆135,624Updated this week
- Durable local inference for Oh My Pi: NInfer + Qwen3.8 27B on one RTX 3090/4090/5090, with restart-resumable OpenAI Responses state.☆51Updated this week
- A local inference engine for Apple silicon, built around the model.☆2,396Updated this week
- Desktop UI for the NInfer local LLM inference engine☆33Nov 2, 2026Updated 2 weeks ago
- Xolotl: run DeepSeek-V4.1-Flash, all 552B, as a local coding agent on one gaming PC (RTX 4090 + 110 GB RAM): ~33 tok/s on agent tasks, ~4…☆535Updated this week
- GVS5H: Five Qwen3.8-27B Models Match Claude Fable 5 on LiveCodeBench Hard | Fable 5 Level Coding for a Fifth the Price - or on a Single…☆1,207Nov 5, 2026Updated last week
- NInfer on Tesla V100 (sm_70): Qwen3.8-27B deployment, build/convert scripts, systemd service, and cross-engine benchmark (vs llama.cpp). …☆56Sep 15, 2026Updated 2 months ago
Are these results useful?