Run Qwen3.8-27B at 140 tok/s on a single RTX 3090, up to 2x llama.cpp. Fast local LLM inference server with OpenAI- and Anthropic-compatible APIs, 256K context, native Windows and Linux.
☆78Nov 12, 2026Updated this week
Alternatives and similar repositories for ninfer-3090
Users that are interested in ninfer-3090 are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- llama.cpp fork for significantly improved performance on Ampere generation Nvidia cards (especially RTX 3090 / 3090 Ti): example: 110+ to…☆371Updated this week
- Multi-GPU (tensor-parallel) CUDA megakernel for Qwen3.8-27B GGUF (Q4_K_M, Q5_K_M, Q6_K, Q8_0) with on-device MTP speculative decoding. Fo…☆16Oct 23, 2026Updated 2 weeks ago
- Qwen3.8-27B EXL3 + DFlash2 speculative decoding on an AMD RX 7900 XTX (exllamav3-rocm): recipe, patches, benchmarks☆23Nov 2, 2026Updated last week
- A custom fork for frontier-scale mixture-of-experts inference on an RTX 3060 with 12 GB VRAM, 16 GB system RAM and local NVMe.** Betelge…☆125Updated this week
- Web front end for making video and music on your own ComfyUI — MiniMax H3, Music 3, SeedVR2 upscaling and FLUX Kontext edits. Deploys to …☆19Updated this week
- Wordpress hosting with auto-scaling - Free Trial Offer • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- Desktop UI for the NInfer local LLM inference engine☆33Nov 2, 2026Updated last week
- An electron Wrapper for Open-Interpreter for the lablab.ai hackathon☆12Oct 14, 2023Updated 3 years ago
- One line of NInfer for the Blackwell, Ada Lovelace, Ampere, Turing and Volta NVIDIA GPUs, consolidated from the forks that carry it: GGUF…☆215Updated this week
- A patching tool to remove the numpy<2 constraint from official mediapipe wheels.☆11Jun 23, 2026Updated 4 months ago
- Browser-based operator console for managing and monitoring Lemonade on Linux inference servers.☆23Updated this week
- An offline coding agent for local models. One command sets up the best model your GPU can run, and on 24 GB+ RTX 30/40/50-series cards Qw…☆141Updated this week
- Pure-Rust inference engine for Qwen GGUF models (Qwen3 to Qwen3.8, dense and MoE up to 770B+). CUDA and Vulkan (NVIDIA, AMD), hybrid CPU+…☆223Updated this week
- Sol-Attn (arXiv 2607.24027) sparse acceleration for MiniMax H3 on V100 — single-node ComfyUI plugin (embedded FP16Safe + keep-or-drop ker…☆34Aug 29, 2026Updated 2 months ago
- Clean AI-generation artifacts from Suno tracks, then master for release. Free, offline, runs entirely on your machine.☆22Sep 25, 2026Updated last month
- Wordpress hosting with auto-scaling - Free Trial Offer • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- vLLM + Qwen3.6-27B (BF16) OpenAI-compatible inference server on AMD Strix Halo (Ryzen AI Max+ 395, gfx1151). Vision input, 256K context, …☆19Apr 26, 2026Updated 6 months ago
- ROCm/AMD GPU benchmark suite for llama.cpp, whisper.cpp, PyTorch☆21Feb 4, 2026Updated 9 months ago
- A strict, auditable HumanEval benchmark runner for GGUF models served via llama.cpp, using its OpenAI-compatible HTTP API.☆14Jan 16, 2026Updated 9 months ago
- A Rust library for draping polygons over terrain with gfx☆11Mar 5, 2018Updated 8 years ago
- A tool to unlobotomize your NVIDIA card!☆31Sep 6, 2026Updated 2 months ago
- ☆12Nov 4, 2026Updated last week
- 开源waf web 防火墙☆10Nov 30, 2021Updated 4 years ago
- WeaponSwingTimer addon for classic, modified by me to fix bugs and add features. Most of the credit goes to LeftHandedGlove for the major…☆14Updated this week
- ☆14Apr 29, 2026Updated 6 months ago
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- An local MITM HTTP(s) request message proxy server☆13Sep 17, 2020Updated 6 years ago
- Bash script presents crontab in human readable form.☆11Apr 21, 2018Updated 8 years ago
- MCP server for World of Warcraft addon development: 20 tools that give Claude, Cursor and any MCP client the in-game Lua API, Blizzard's …☆27Updated this week
- A open source desktop app for building Movies with Minimax H3.☆52Oct 28, 2026Updated 2 weeks ago
- Client Apps for AURGA Viewer☆13Oct 5, 2026Updated last month
- 📱 ASO & Mobile App Growth Playbook — Complete App Store Optimization guide. Cold start strategies, UGC creator operations, TikTok/Instag…☆13May 7, 2026Updated 6 months ago
- A fast and stealthy credential harvester☆11Aug 14, 2015Updated 11 years ago
- Bloat Free, Portable and Lightweight LLM Frontend (Single HTML file). With Lorebook, Web Search, Macro Engine etc.☆23Updated this week
- 相册语义浏览器：搜索、地图、聚类、人物识别、收藏、时间线 | Semantic album browser with search, map, clustering, face recognition & timeline☆17Jun 7, 2026Updated 5 months ago
- 1-Click AI Models by DigitalOcean Gradient • AdDeploy popular AI models on DigitalOcean Gradient GPU virtual machines with just a single click. Zero configuration with optimized deployments.
- A .NET client for Websockets without SignalR☆19Mar 23, 2020Updated 6 years ago
- 170hx-pve-gen2-controller☆11Aug 20, 2026Updated 2 months ago
- High-performance single-GPU inference for selected model checkpoints and GPUs.☆24Oct 13, 2026Updated 3 weeks ago
- Diagnose vLLM inference servers☆18Jul 31, 2026Updated 3 months ago
- A canvas for working things out. Your agent writes live TSX frames into your repo, and you play the flows between them like a real app.☆23Updated this week
- ☆14Oct 20, 2026Updated 3 weeks ago
- Raspberry Pi Pico Relay Board is developed by SB Components with the potential to control up to 4 appliances and loads up to 240V AC@ 7A,…☆13Mar 26, 2024Updated 2 years ago
Are these results useful?