Rulith Inference, formerly Strix Llama. Serving a 125B MoE model fast on one AMD Strix Halo machine, on Windows: a patched llama.cpp, a manager, and a desktop app.
☆37Oct 16, 2026Updated this week
Alternatives and similar repositories for rulith-inference
Users that are interested in rulith-inference are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- A fast diffusion model engine for image generation and editing.☆35Updated this week
- Portable optimized ROCm and Qwen3.8-27B setup for AMD Strix Halo☆83Sep 16, 2026Updated last month
- LLM inference in C/C++☆42Updated this week
- GPU + XDNA2 NPU inference for AMD Strix Halo (gfx1151): Qwen3.6-35B-A3B, Qwen3.8-27B and Qwen3.8 Flash-Next. NPU prefill +9% to +28%, MTP…☆58Updated this week
- Atlas Inference is a dependency free lightweight inference engine meant to run LLMs on YOUR compute seamlessly. Supports GB10 and Strix H…☆34Updated this week
- Bare Metal GPUs on DigitalOcean Gradient AI • AdPurpose-built for serious AI teams training foundational models, running large-scale inference, and pushing the boundaries of what's possible.
- Docker Compose for llama.cpp GGUF servers on AMD Strix Halo: Qwen, Gemma, and Laguna packages (abliterated and quantized), stock Vulkan p…☆27Aug 17, 2026Updated 2 months ago
- High-performance Lemonade alternative for AMD Strix Halo & Radeon — latest MoE models (Ling-3.0-Flash, Qwen 3.8 Flash Next, Ornith 1.5), …☆90Sep 24, 2026Updated 3 weeks ago
- Performance-tuned llama.cpp for AMD Strix Halo (gfx1151): FA + MoE-prefill fixes with a bundled current Mesa driver. Vulkan and HIP; port…☆217Oct 9, 2026Updated last week
- The Open Source Strix Halo AI engine. Qwen 3.8 Flash Next Q4_K_XL: 1,801.01 pp, 73.17 tg single user, 194.37 tok/s 8 users; Qwen 3.8 27B…☆811Updated this week
- llama.cpp modified to work with Strix Halo + dGPU as an accelerator, Dual Strix Halo Machines, or both☆51Updated this week
- Qwen and DeepSeek low-VRAM LoRA training with GGUF base model, compatible with Transformers and PEFT☆22Updated this week
- llama.cpp fork tuned for Qwen 3.8 Flash-Next on AMD Strix Halo — RDNA3.5 kernel patches, true QSA sparse gather, working MTP, 27 GB engra…☆50Updated this week
- The fastest way to run Qwen3.8-Flash-Next on Strix Halo (gfx1151)☆1,018Updated this week
- LLM inference engine written from scratch in HIP for AMD Strix Halo (gfx1151) and Qwen3.8-Flash-Next - ~50 tokens/s out to 492k context☆62Updated this week
- Proton VPN Special Offer - Get 70% off • AdSpecial partner offer. Trusted by over 100 million users worldwide. Tested, Approved and Recommended by Experts.
- Qwen 3.8 27B ROCmFP4 on AMD Strix Halo (Ryzen AI Max+ 395). Up to 36 tok/s via MTP Speculation, TurboQuant & Mesa RADV Wave64.☆258Sep 7, 2026Updated last month
- LLM inference in C/C++☆236Updated this week
- ☆89Updated this week
- Lightweight Python tool using Optuna for tuning llama.cpp flags: towards optimal tok/s for your machine☆60Aug 17, 2026Updated 2 months ago
- ☆21Aug 17, 2026Updated 2 months ago
- Reverse engineering, documentation, and experimental firmware development for the original Polyend Tracker.☆42Updated this week
- Qwen3.8-Flash-Next on any consumer hardware: one-click install for Windows / Linux. Strata inference engine, OpenAI/Anthropic API on loca…☆25Updated this week
- A beautifully crafted desktop client for running and managing local LLMs via llama.cpp.☆23May 25, 2026Updated 4 months ago
- Container toolbox that runs llama.cpp with AMD ROCm and NVIDIA CUDA in one build, for inference on two GPUs☆20Updated this week
- End-to-end encrypted email - Proton Mail • AdSpecial offer: 40% Off Yearly / 80% Off First Month. All Proton services are open source and independently audited for security.
- RocM Optimized ComfyUI nodes☆51Updated this week
- Goal Driven Development - Transform software development from task execution to outcome achievement with goal-driven development methodo…☆14Oct 1, 2026Updated 2 weeks ago
- LLM inference in C/C++☆34Aug 1, 2026Updated 2 months ago
- Kotai is a fully local, zero-cost voice assistant that combines the power of Kyutai TTS/STT, LiveKit, and local LLMs to create natural co…☆10Jul 14, 2025Updated last year
- Simple Tool Caller for llama.cpp☆11Aug 12, 2024Updated 2 years ago
- Windows desktop console for llama.cpp runtimes, models, and local coding workflows☆132Updated this week
- Ion is a zero-install, browser-native coding agent.☆41Updated this week
- Kyojin: the Yamz inference engine for AMD Strix Halo (ROCm, gfx1151), built on ExLlamaV3. Runs 300B-class MoE models on one 128 GB mini P…☆194Updated this week
- The $0 AI Empire Playbook — 180 scheduled jobs, 43 free LLM lanes, and the failure classes that almost killed it. Free tiers only.☆101Sep 22, 2026Updated 3 weeks ago
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- Pure C wrapper library to use llama.cpp with Linux and Windows as simple as possible.☆15Sep 6, 2026Updated last month
- FFMPEG For Delphi☆33Jul 27, 2023Updated 3 years ago
- ☆17Apr 7, 2024Updated 2 years ago
- Browse Civitai workflows & models inside ComfyUI and get any workflow ready in one click: load it, find missing models and download them …☆26Oct 3, 2026Updated 2 weeks ago
- AI SDK provider for Google's Gemini models with built-in OAuth authentication. Access Gemini 2.5 Pro/Flash for free using the same auth f…☆18Aug 27, 2025Updated last year
- The fastest way to run Qwen3.8 27B on Strix Halo (gfx1151)☆133Sep 14, 2026Updated last month
- Run Tencent's HunyuanImage-3.0 (80B MoE) natively in ComfyUI on a 24 GB GPU: base, Instruct and Instruct-Distil, prompt rewriting, image …☆41Oct 9, 2026Updated last week