Qwen3.8-27B on one RTX 4090: full native 262K context via E8 4-bit KV, up to 149 tok/s code decode with MTP3, sm_89-retuned attention prefill, vision, llama.cpp-compatible /metrics + /slots
☆239Oct 31, 2026Updated 2 weeks ago
Alternatives and similar repositories for ninfer-4090
Users that are interested in ninfer-4090 are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- NInfer port for Qwen 3.8 27B on RTX 4090☆187Updated this week
- Qwen3.8-27B on one RTX 4090: full native 262K context via E8 4-bit KV, up to 149 tok/s code decode with MTP3, sm_89-retuned attention pre…☆15Sep 29, 2026Updated last month
- High-performance single-GPU inference for selected model checkpoints and GPUs.☆3,379Updated this week
- ☆36Updated this week
- Hyper optimised Qwen3.8-27B inference on one RTX 3090: ReplaySSM, MTP3, reasoning effort, C1-C8 batching, and native Windows and Linux bu…☆523Oct 31, 2026Updated 2 weeks ago
- 1-Click AI Models by DigitalOcean Gradient • AdDeploy popular AI models on DigitalOcean Gradient GPU virtual machines with just a single click. Zero configuration with optimized deployments.
- One line of NInfer for the Blackwell, Ada Lovelace, Ampere, Turing and Volta NVIDIA GPUs, consolidated from the forks that carry it: GGUF…☆232Updated this week
- Qwen3.8-27B on a single RTX 4090 (sm_89): Cinference/NInfer port with MTP speculative decoding and K8V4 KV cache☆15Nov 1, 2026Updated 2 weeks ago
- Qwen3.8-27B-GSQ3 on one RTX 4080: 100k context with rk4v4-e8, up to 2720 tok/s prefill and 262 tok/s generation☆34Oct 9, 2026Updated last month
- GPU tensor unlock source for RTX 4090 and experimental RTX 4070 Ti SUPER; firmware generated locally☆4,934Oct 26, 2026Updated 3 weeks ago
- Qwen4-Exp (Qwen3.8-Flash-Next) SWA + MTP + PLE-on-iswa — bounded deep-context decode w/ long-range recall, TBQ4 4.25 bpv KV, DSpark + Rot…☆94Sep 24, 2026Updated last month
- ☆11Sep 23, 2024Updated 2 years ago
- An OpenAI API compatible moderations server for checking whether text is potentially harmful.☆10Mar 23, 2024Updated 2 years ago
- Windows desktop console for llama.cpp runtimes, models, and local coding workflows☆143Updated this week
- More MoEs: a fork of Strata (Qwen3.8-Flash-Next, Qwen3.6/Ornith and more on one GPU + system RAM)☆27Updated this week
- Managed Kubernetes at scale on DigitalOcean • AdDigitalOcean Kubernetes includes the control plane, bandwidth allowance, container registry, automatic updates, and more for free.
- Run GLM-5.3 (744B), GLM-5.3-Flash (321B), DeepSeek-V4.1-Flash (748B) and MiniMax-M3.1-Flash (428B) on your own GPUs - one engine, one das…☆3,018Updated this week
- An OpenAI API compatible images server to generate or manipulate images.☆18Feb 2, 2025Updated last year
- ☆19Aug 10, 2026Updated 3 months ago
- Serve large Qwen models fast on the GPUs you actually own. Qwen3.8-27B on a single 24 GB card with vLLM: 127 tok/s single-user (381 when …☆2,751Nov 10, 2026Updated last week
- Unlock FP/BF16 instruction issue rate on Nvidia 3000's cards to their professional equivalents☆42Oct 23, 2026Updated 3 weeks ago
- Memory system for LLMs that remembers everything you teach it during conversation. No reindexing, no context window limits. CPU by defaul…☆13May 2, 2026Updated 6 months ago
- KVarN, KV cache precision tail, low-bit quants in llama.cpp for longer context of better precision in the same VRAM☆1,197Updated this week
- Visual JS Code tests for visual benchmarking local LLM capabilities☆11Sep 18, 2026Updated 2 months ago
- PCIe P2P on consumer RTX 5090 via one GSP firmware registry key — no driver patches.☆25Apr 17, 2026Updated 7 months ago
- Deploy open-source AI quickly and easily - Special Bonus Offer • AdRunpod Hub is built for open source. One-click deployment and autoscaling endpoints without provisioning your own infrastructure.
- ☆18Apr 10, 2026Updated 7 months ago
- monitor for a local LLM runtime (llama-swap, NInfer, Strata, more) -- metrics ported to berth.☆16Oct 17, 2026Updated last month
- Durable local inference for Oh My Pi: NInfer + Qwen3.8 27B on one RTX 3090/4090/5090, with restart-resumable OpenAI Responses state.☆51Updated this week
- Qwen3.8-Flash-Next IQ3_S on 12 GB VRAM + 32 GB RAM + NVMe: ~45-50 tok/s decode, 131k context, Strata unify fork☆21Oct 24, 2026Updated 3 weeks ago
- A standalone, modular prompt generator for ComfyUI. Generate detailed, natural language prompts for characters and spaceships with extens…☆14Aug 3, 2026Updated 3 months ago
- MiMo-V2.6-Flash-RL on 2x NVIDIA DGX Spark (GB10): vLLM TP2 + DFlash + vision/audio, fp8 KV 300K, four fixes for the stock image, measured…☆47Oct 4, 2026Updated last month
- Read-Only source code mirror, Proxmox uses mailing list workflow for development.☆27Aug 21, 2026Updated 2 months ago
- Scripts to force Davinci Resolve to reload the latest version of media files☆14Jan 5, 2025Updated last year
- Extension for Sequential Image Inpainting Available in ComfyUI☆48Jun 29, 2025Updated last year
- AI Agents on DigitalOcean Gradient AI Platform • AdBuild production-ready AI agents using customizable tools or access multiple LLMs through a single endpoint. Create custom knowledge bases or connect external data.
- ☆16Oct 2, 2026Updated last month
- ComfyUI MiniMax H3 安全缓存加速节点|Bilingual safe residual-cache acceleration node☆13Nov 5, 2026Updated 2 weeks ago
- Model-agnostic ledger of avoidable agent token spend: name why prompt cache misses, price the miss, prove fixes without losing task succe…☆27Updated this week
- An all-in-one, pure C++ inference engine for audio models, powered by ggml. Supports TTS, STT, VAD, voice conversion, music generation, a…☆4,318Updated this week
- llama.cpp fork with additional SOTA quants and improved performance☆3,403Updated this week
- A tool to unlobotomize your NVIDIA card!☆925Oct 28, 2026Updated 3 weeks ago
- Tensor-Core selector unlock for RTX 4070 Ti Super☆28Oct 20, 2026Updated 3 weeks ago
Are these results useful?