Qwen3.8-Flash-Next (MoE) and Qwen3.8-27B on one or two 16 GB RTX 50-series GPUs: experts offloaded to CPU, two-GPU tensor parallel. 64 GB of RAM is enough for Flash-Next. Formerly ninfer-tp2.
☆140Nov 8, 2026Updated this week
Alternatives and similar repositories for capy-flash
Users that are interested in capy-flash are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Strata fork: Qwen3.8-Flash-Next 125B MoE in NVFP4 on one RTX 20-50 card (12 GB+) and 64 GB of RAM or more. Setup installs our GPTQ quants…☆228Updated this week
- High-performance single-GPU inference for selected model checkpoints and GPUs.☆16Aug 29, 2026Updated 2 months ago
- Local inference engine for Qwen3.8-Flash-Next on NVIDIA Blackwell☆822Updated this week
- Multi-GPU prefill acceleration for llama.cpp☆16Oct 23, 2026Updated 2 weeks ago
- Huggingface but for inference engines.☆14Oct 31, 2026Updated last week
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- Berth is a lightweight GPU resource manager written in Go. It controls which applications get access to your GPU, when they get it, and h…☆291Updated this week
- An offline coding agent for local models. One command sets up the best model your GPU can run, and on 24 GB+ RTX 30/40/50-series cards Qw…☆139Nov 1, 2026Updated last week
- Unlock P2P comms between consumer NVIDIA GPUs☆702Oct 26, 2026Updated 2 weeks ago
- Source-available C++/CUDA inference engine for Qwen3.8-27B INT4 on 3x RTX 3090☆44Updated this week
- Multi-dataset evaluation of Anthropic's J-Space on Qwen3-4B. Demonstrates that internal 'workspace noise' can reliably route high-confide…☆12Jul 13, 2026Updated 3 months ago
- Run GLM-5.3-Flash (321B), DeepSeek-V4.1-Flash (748B) and MiniMax-M3.1-Flash (428B) on your own GPUs - one engine, one dashboard, OpenAI- …☆2,727Updated this week
- C++/CUDA single-GPU inference engine for Qwen3.8 (incl. Qwen3.8-Flash-Next with offloaded experts) on the RTX 5090; grew from NInfer☆284Updated this week
- A fork of https://github.com/Niko1221/Strata optimized for my 3090+5070 Ti setup, see docs/COMPARISON.md. Renamed to Lamina.☆60Updated this week
- A NPC AI Framework , based on character personalities, can be used for game NPC (extendable for Godot / Unity)☆23Sep 21, 2026Updated last month
- Managed Kubernetes at scale on DigitalOcean • AdDigitalOcean Kubernetes includes the control plane, bandwidth allowance, container registry, automatic updates, and more for free.
- ChatGPT-style generative UI for Open WebUI: photo galleries, live maps with real routes, multi-day trips, budgets, calculators, live weat…☆66Updated this week
- Container toolbox that runs llama.cpp with AMD ROCm and NVIDIA CUDA in one build, for inference on two GPUs☆22Oct 16, 2026Updated 3 weeks ago
- Run MoE models bigger than your RAM, without the speed cliff☆21Updated this week
- Pure-Rust inference engine for Qwen GGUF models (Qwen3 to Qwen3.8, dense and MoE up to 80B+). CUDA and Vulkan (NVIDIA, AMD), hybrid CPU+G…☆159Updated this week
- Xolotl: run DeepSeek-V4.1-Flash, all 552B, as a local coding agent on one gaming PC (RTX 4090 + 110 GB RAM): ~33 tok/s on agent tasks, ~4…☆495Updated this week
- Stable, high-throughput inference server for multi-GPU NVIDIA V100 (SM70): Qwen3.8 Flash-Next with AWQ / NVFP4, multimodal input, CPU/dis…☆49Updated this week
- It lets an AI agent put a live, interactive page on any screen you own, and brings what you do on it back into the conversation.☆26Updated this week
- llama.cpp fork for significantly improved performance on Ampere generation Nvidia cards (especially RTX 3090 / 3090 Ti): example: 110+ to…☆302Updated this week
- A Raspberry Pi project that listens to audio from a USB turntable, identifies the currently playing song using ShazamIO, generates a 64×6…☆41Updated this week
- AI Agents on DigitalOcean Gradient AI Platform • AdBuild production-ready AI agents using customizable tools or access multiple LLMs through a single endpoint. Create custom knowledge bases or connect external data.
- KVarN, KV cache precision tail, low-bit quants in llama.cpp for longer context of better precision in the same VRAM☆1,188Updated this week
- LLM inference in C/C++☆12Oct 18, 2026Updated 3 weeks ago
- WMMA GEMM in ROCm for RDNA GPUs☆20Sep 1, 2026Updated 2 months ago
- Local AI for your whole house: chat, images and voice on your own GPU. Computes each model's flags, fits them to your VRAM, and hot-swaps…☆33Updated this week
- Unity package repo for UIMan☆14Mar 27, 2021Updated 5 years ago
- High-performance single-GPU inference for selected model checkpoints and GPUs.☆3,293Updated this week
- Known-good vLLM recipe: Qwen3.8-27B-Int8 + MTP speculative decoding on NVIDIA CMP 170HX 64GB (Hynix) — 84 tok/s, 262k ctx☆13Oct 8, 2026Updated last month
- GunBound Classic (Thor's Hammer) emulator☆13Jul 18, 2026Updated 3 months ago
- Example of BBSNetworkSystem☆15Jul 8, 2018Updated 8 years ago
- Managed Database hosting by DigitalOcean • AdPostgreSQL, MySQL, MongoDB, Kafka, Valkey, and OpenSearch available. Automatically scale up storage and focus on building your apps.
- Run Qwen3.8-Flash-Next on one AMD Strix Halo (Ryzen AI Max+ 395, gfx1151). EXL3 4.05 and 3.05 bpw packs, own ROCm/HIP kernels, OpenAI-com…☆19Updated this week
- SNDR Core Engine (Genesis) — vLLM runtime patch-overlay for Qwen3.6 + Gemma4 on consumer NVIDIA (Ampere sm_86, 2× A5000/3090). Qwen3.6-35…☆133Sep 30, 2026Updated last month
- A beautifully crafted desktop client for running and managing local LLMs via llama.cpp.☆24May 25, 2026Updated 5 months ago
- Run local LLMs on a Tesla V100-SXM2-16GB: prebuilt binaries, serve scripts, and setup for Windows and Linux (Qwen3 + Gemma 4, Claude Code…☆20Sep 1, 2026Updated 2 months ago
- A .NET library for image resizing at different qualities with very high quality support.☆11Apr 23, 2012Updated 14 years ago
- Typed choices and probability distributions from text and images with llama.cpp, Ollama, and OpenRouter☆24Updated this week
- Generative UI for Hermes Agent: charts, tables, plans, calculators and explainers drawn right in Hermes Desktop chat replies☆26Updated this week
Are these results useful?