llama.cpp/ik_llama.cpp launcher: loads big MoE models across mismatched multi-GPU rigs by exact VRAM math.
☆284Nov 19, 2026Updated this week
Alternatives and similar repositories for ggrun
Users that are interested in ggrun are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Linux & Powershell scripts to easily set up and run the Qwen 3.5 series locally on Windows and Linux with llama.cpp.☆115Nov 3, 2026Updated 2 weeks ago
- KVarN, KV cache precision tail, low-bit quants in llama.cpp for longer context of better precision in the same VRAM☆1,199Updated this week
- SNDR Core Engine (Genesis) — vLLM runtime patch-overlay for Qwen3.6 + Gemma4 on consumer NVIDIA (Ampere sm_86, 2× A5000/3090). Qwen3.6-35…☆133Sep 30, 2026Updated last month
- Fast open-source (MIT) LLM inference engine in C++/CUDA for RTX 3090-class GPUs: Qwen3.8-Flash-Next (~1,000 tok/s with 8 agents on 3x RTX…☆75Updated this week
- An optimized quantization and inference library for running LLMs locally on modern consumer-class GPUs☆1,846Updated this week
- Serverless GPU API endpoints on Runpod - Get Bonus Credits • AdSkip the infrastructure headaches. Auto-scaling, pay-as-you-go, no-ops approach lets you focus on innovating your application.
- Reliable model swapping for any local AI server. Chosen by thousands of users. Batteries included. If we don't have it, you probably don…☆6,281Updated this week
- Proxy for OpenAI☆16Sep 2, 2025Updated last year
- Community recipes for serving LLMs on RTX 3090/4090/5090 CUDA gpus. Multi-engine (vLLM, sglang, llama.cpp and other custom engines) and m…☆2,507Updated this week
- Krasis is a Hybrid LLM runtime which focuses on efficient running of larger models on consumer grade VRAM limited hardware☆531Nov 12, 2026Updated last week
- Lucebox inference engine for heterogeneous hardware & consumer GPUs☆3,023Updated this week
- Specialized fork for (relatively) fast single-GPU inference (in CUDA) using large MoE models that don't fit fully into VRAM☆17May 6, 2026Updated 6 months ago
- ik_llama.cpp's Thireus fork with release builds for macOS/Windows/Ubuntu CPU, Vulkan and CUDA☆185Updated this week
- Open-source framework for superagents.☆112Sep 11, 2026Updated 2 months ago
- Build a software startup without writing code. Not vibe coding: a proven team workflow (requirements, design, test-first build, QA, relea…☆16Oct 30, 2026Updated 3 weeks ago
- Proton VPN Special Offer - Get 70% off • AdSpecial partner offer. Trusted by over 100 million users worldwide. Tested, Approved and Recommended by Experts.
- ☆21Updated this week
- Human-AI Document Standard — lightweight convention for AI-optimized technical documentation☆28Jul 9, 2026Updated 4 months ago
- ☆43May 4, 2026Updated 6 months ago
- Pi compaction that reuses the server's prefix cache (vLLM / SGLang / llama.cpp Anthropic endpoints): no cold re-prefill, plus a warm-up s…☆36Nov 6, 2026Updated 2 weeks ago
- ☆23Feb 17, 2026Updated 9 months ago
- An offline coding agent for local models. One command sets up the best model your GPU can run, and on 24 GB+ RTX 30/40/50-series cards Qw…☆147Updated this week
- ☆18Nov 4, 2026Updated 2 weeks ago
- handwritten harness for AI, not vibecoded, everything is a module/plugin, tailor made for use with local models, tiny system prompt (arou…☆544Updated this week
- Llama.cpp launcher with integrated huggingface☆77Oct 14, 2026Updated last month
- End-to-end encrypted email - Proton Mail • AdSpecial offer: 40% Off Yearly / 80% Off First Month. All Proton services are open source and independently audited for security.
- Run autoresearch on any NVIDIA GPUs (Works on 2-4GB+ Cards)☆27Mar 22, 2026Updated 7 months ago
- A bare-bones GUI application for the local inference engine, llama.cpp. Built-in TPE optimiser to find the best flags for your system☆18Oct 15, 2026Updated last month
- Produce your own Dynamic 3.0 Quants and achieve optimum accuracy & SOTA quantization performance! Input a target size and the toolchain w…☆174Sep 11, 2026Updated 2 months ago
- RDNA-native LLM inference engine in Rust.☆821Updated this week
- Run huge AI models on your own PC, even ones bigger than your memory.☆75Updated this week
- A harness optimized to smaller LLMs☆2,735Sep 18, 2026Updated 2 months ago
- A lightweight, prompt-driven MCP web research server for high-quality LLM powered information extraction.☆138Apr 10, 2026Updated 7 months ago
- An ongoing, collaborative meta-analysis about Human-AI-Interactions. We aggregate data and knowledge to build a non-abrasive, user-friend…☆116Aug 25, 2026Updated 2 months ago
- An agent that builds agents — closed-loop AI system that interviews, designs, tests, and ships standalone Python agents.☆14Jul 29, 2026Updated 3 months ago
- GPU virtual machines on DigitalOcean Gradient AI • AdGet to production fast with high-performance AMD and NVIDIA GPUs you can spin up in seconds. The definition of operational simplicity.
- the composable multi-agent shell☆506Updated this week
- Open-source deep-work orchestrator: a frontier planner proposes one small epoch at a time; local models grind; deterministic gates verify…☆41Jul 11, 2026Updated 4 months ago
- Thinking blocks that open during generation: a gate decides when to think, a tree of candidate thoughts is explored, and the best one sta…☆18Oct 11, 2026Updated last month
- Llama.cpp runner/swapper and proxy that emulates LMStudio / Ollama backends☆61Aug 21, 2025Updated last year
- Profile-guided GPU kernel optimizer for AMD/RDNA3. Auto-tunes llama.cpp MMVQ kernels per model shape. 2x decode speedup on 7900 XTX.☆71Sep 5, 2026Updated 2 months ago
- SmarterRouter: An intelligent LLM gateway and VRAM-aware router for Ollama, llama.cpp, and OpenAI. Features semantic caching, model profi…☆164May 10, 2026Updated 6 months ago
- Fulloch - The Fully Local Home Voice Assistant☆176Nov 13, 2026Updated last week
Are these results useful?