Qwen3.8-27B on one RTX 4090: full native 262K context via E8 4-bit KV, up to 149 tok/s code decode with MTP3, sm_89-retuned attention prefill, vision, llama.cpp-compatible /metrics + /slots
☆222Oct 26, 2026Updated this week
Alternatives and similar repositories for ninfer-4090
Users that are interested in ninfer-4090 are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- NInfer port for Qwen 3.8 27B on RTX 4090☆183Sep 9, 2026Updated last month
- Qwen3.8-27B on one RTX 4090: full native 262K context via E8 4-bit KV, up to 149 tok/s code decode with MTP3, sm_89-retuned attention pre…☆15Sep 29, 2026Updated last month
- High-performance single-GPU inference for selected model checkpoints and GPUs.☆3,206Updated this week
- NInfer-4090: NInfer inference engine ported to RTX 4090 (sm_89) with YaRN support for Qwen3.8-27B☆29Oct 2, 2026Updated 3 weeks ago
- Desktop UI for the NInfer local LLM inference engine☆32Oct 20, 2026Updated last week
- GPU virtual machines on DigitalOcean Gradient AI • AdGet to production fast with high-performance AMD and NVIDIA GPUs you can spin up in seconds. The definition of operational simplicity.
- Hyper optimised Qwen3.8-27B inference on one RTX 3090: ReplaySSM, MTP3, reasoning effort, C1-C8 batching, and native Windows and Linux bu…☆484Oct 16, 2026Updated last week
- Qwen3.8-27B-GSQ3 on one RTX 4080: 100k context with rk4v4-e8, up to 2720 tok/s prefill and 262 tok/s generation☆30Oct 9, 2026Updated 2 weeks ago
- ☆11Sep 23, 2024Updated 2 years ago
- Windows desktop console for llama.cpp runtimes, models, and local coding workflows☆134Updated this week
- PCIe P2P on consumer RTX 5090 via one GSP firmware registry key — no driver patches.☆18Apr 17, 2026Updated 6 months ago
- Qwen4-Exp (Qwen3.8-Flash-Next) SWA + MTP + PLE-on-iswa — bounded deep-context decode w/ long-range recall, TBQ4 4.25 bpv KV, DSpark + Rot…☆95Sep 24, 2026Updated last month
- Project Maya - run GLM-5.3-Flash (321B MoE), DeepSeek-V4.1-Flash-1017 (748B MoE) and MiniMax-M3.1-Flash (428B MoE) privately on your own …☆2,190Updated this week
- Deployment kit: Qwen3.8-27B EXL3 3.5bpw target + DFlash2 EXL3 5.0bpw speculative draft — launcher, config, model cards☆233Sep 1, 2026Updated last month
- Qwen3.8-Flash-Next on any consumer hardware: one-click install for Windows / Linux. Strata inference engine, OpenAI/Anthropic API on loca…☆35,851Updated this week
- AI Agents on DigitalOcean Gradient AI Platform • AdBuild production-ready AI agents using customizable tools or access multiple LLMs through a single endpoint. Create custom knowledge bases or connect external data.
- Durable local inference for Oh My Pi: NInfer + Qwen3.8 27B on one RTX 3090/4090/5090, with restart-resumable OpenAI Responses state.☆47Updated this week
- Serve large Qwen models fast on the GPUs you actually own. Qwen3.8-27B on a single 24 GB card with vLLM: 127 tok/s single-user (381 when …☆2,413Oct 21, 2026Updated last week
- One line of NInfer for the Blackwell, Ada Lovelace, Ampere, Turing and Volta NVIDIA GPUs, consolidated from the forks that carry it: GGUF…☆167Updated this week
- ComfyUI MiniMax H3 全家桶：Unified to Video / Context IR / Multimodal Chat / Audio Lock / Media Loader / Resolution Selector / Concat AV 等 8 …☆42Oct 20, 2026Updated last week
- ninfer-ext: Forked for speed.☆27Oct 22, 2026Updated last week
- GPU tensor unlock source for RTX 4090 and experimental RTX 4070 Ti SUPER; firmware generated locally☆4,485Updated this week
- A standalone, modular prompt generator for ComfyUI. Generate detailed, natural language prompts for characters and spaceships with extens…☆15Aug 3, 2026Updated 2 months ago
- A tool to unlobotomize your NVIDIA card!☆815Updated this week
- Extension for Sequential Image Inpainting Available in ComfyUI☆48Jun 29, 2025Updated last year
- Managed Kubernetes at scale on DigitalOcean • AdDigitalOcean Kubernetes includes the control plane, bandwidth allowance, container registry, automatic updates, and more for free.
- We gave AI models telepathy.☆21Sep 25, 2026Updated last month
- Decides who gets your GPU. Chat, image and voice engines lease the card in turn instead of grabbing it, so two of them never load at once…☆258Updated this week
- Xolotl (formerly Strata-DSV4.1): run DeepSeek-V4.1-Flash, all 552B, as a local coding agent on one gaming PC (RTX 4090 + 90 GB RAM) at up…☆404Updated this week
- A temporal video upscaler based on SPAN☆12Sep 11, 2026Updated last month
- KVarN, KV cache precision tail, low-bit quants in llama.cpp for longer context of better precision in the same VRAM☆1,177Updated this week
- Hermes Android mobile client for self-hosted Hermes Gateway☆47Oct 9, 2026Updated 3 weeks ago
- Demos in various langauges for Voxscript☆27Nov 15, 2023Updated 2 years ago
- Unofficial modern-Windows preservation installer for MechWarrior 4 using user-supplied original media☆26Updated this week
- SlopCodeBench problems☆13Oct 7, 2026Updated 3 weeks ago
- AI Agents on DigitalOcean Gradient AI Platform • AdBuild production-ready AI agents using customizable tools or access multiple LLMs through a single endpoint. Create custom knowledge bases or connect external data.
- Comfy UI nodes for Z-Image-Turbo to run native weights☆15Nov 28, 2025Updated 11 months ago
- Easy-to-use dynamic Prompt and LoRA stacker nodes for ComfyUI.☆22Oct 10, 2026Updated 2 weeks ago
- Lightweight Claude Code Telegram AI secretary with reincarnation memory☆18Apr 16, 2026Updated 6 months ago
- Process text and images with GGUF LLMs in ComfyUI using llama.cpp, including Qwen3-VL, Qwen3.5, Qwen3.6, Qwen3.7, Qwen3.8, Gemma 4, and g…☆60Aug 18, 2026Updated 2 months ago
- The fastest inference engine for Qwen3.8-27B on the NVIDIA RTX 5090: up to 500 tokens/s for one agent and up to 2,000 tokens/s for many, …☆51Updated this week
- Plug n Pi Server: USB-connect your phone to look up Raspberry Pi's IP address☆10Feb 3, 2019Updated 7 years ago
- A low-latency GPU inference runtime for streaming GPT-SoVITS☆45Oct 21, 2026Updated last week