Caching experts made model agnostic for llama. Drastically speeds up generation speed on MoE models that fully fit in the RAM but partially in Vram. WIP: Same speedup but for models that don't fit in the ram either.
☆29Aug 6, 2026Updated 2 months ago
Alternatives and similar repositories for llama-wackMall
Users that are interested in llama-wackMall are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- ☆21Aug 17, 2026Updated 2 months ago
- ☆31Aug 26, 2026Updated last month
- ☆27Apr 30, 2026Updated 5 months ago
- LLM inference in C/C++☆34Aug 1, 2026Updated 2 months ago
- hummingbird is a lightweight, zero dependency runtime for massive open source Mixture of Experts (MoE) language models. It unifies SSD, R…☆60Jul 31, 2026Updated 2 months ago
- End-to-end encrypted email - Proton Mail • AdSpecial offer: 40% Off Yearly / 80% Off First Month. All Proton services are open source and independently audited for security.
- Kindle Package Manager☆33Sep 7, 2026Updated last month
- Interactive Jacobian-Lens visualizer and live steerer for GGUF models on llama.cpp☆90Jul 12, 2026Updated 3 months ago
- OpenCode plugin: Ralph outer loop + RLM inner loop — iterative AI development with file-first discipline and sub-agent support☆32Jun 30, 2026Updated 3 months ago
- High-performance single-GPU inference for selected model checkpoints and GPUs.☆24Oct 13, 2026Updated last week
- Qwen3.8-Flash-Next on any consumer hardware: one-click install for Windows / Linux. Strata inference engine, OpenAI/Anthropic API on loca…☆25Updated this week
- Containerized bridges to interconnect services in matrix☆11Updated this week
- ☆14Oct 6, 2026Updated 2 weeks ago
- LLM inference in C/C++☆17Updated this week
- The fastest inference engine for Qwen3.8-27B on the NVIDIA RTX 5090: up to 500 tokens/s for one agent and up to 2,000 tokens/s for many, …☆37Oct 7, 2026Updated last week
- Managed Database hosting by DigitalOcean • AdPostgreSQL, MySQL, MongoDB, Kafka, Valkey, and OpenSearch available. Automatically scale up storage and focus on building your apps.
- Local-first AI workflow orchestration for chaining models, agents, tools, and scripts into repeatable workflows.☆34Aug 10, 2026Updated 2 months ago
- Graphic Devices for R based on NanoVG☆10Jun 15, 2018Updated 8 years ago
- Web search tool for the pi coding agent — batch queries, ranked readable markdown results ready to fetch. Pairs with pi-smart-fetch.☆23Oct 10, 2026Updated last week
- Kotai is a fully local, zero-cost voice assistant that combines the power of Kyutai TTS/STT, LiveKit, and local LLMs to create natural co…☆10Jul 14, 2025Updated last year
- A modern, Qt-based graphical user interface for the USB/IP protocol on Windows. This client allows you to easily connect to remote USB/IP…☆43Oct 8, 2026Updated last week
- Self contained code that can be pulled into an instance running glance - such as an LXC or VM, maybe more☆16Feb 24, 2026Updated 7 months ago
- ☆12Dec 11, 2022Updated 3 years ago
- PCG like for godot☆75Oct 8, 2026Updated last week
- Rulith Inference, formerly Strix Llama. Serving a 125B MoE model fast on one AMD Strix Halo machine, on Windows: a patched llama.cpp, a m…☆37Updated this week
- AI Agents on DigitalOcean Gradient AI Platform • AdBuild production-ready AI agents using customizable tools or access multiple LLMs through a single endpoint. Create custom knowledge bases or connect external data.
- A local-first coding agent that doesn't take "done" on faith: the engine keeps the run going until the project's own tests pass, and ever…☆82Sep 22, 2026Updated 3 weeks ago
- Harn is a programming language and runtime for building AI agents.☆27Updated this week
- SOLID principles and design patterns for backend code — applied with restraint. Skill for Claude Code, OpenCode, GitHub Copilot, and Open…☆41Jul 14, 2026Updated 3 months ago
- Merged into the Amecs project: https://github.com/Siphalor/amecs☆15Feb 22, 2026Updated 7 months ago
- ENGRAFT: write facts into an LLM's n-gram (Engram) memory table as a removable overlay: 100 facts in a 9 MB file, no weights touched, ver…☆57Sep 30, 2026Updated 2 weeks ago
- Enable full DualSense/ DualSense Edge features (Adaptive Triggers, HD Haptics, Gyro, Touchpad) over Moonlight/Sunshine game streaming via…☆38May 12, 2026Updated 5 months ago
- Qwen3.8-Flash-Next 180B NVFP4 @ 231 tok/s on one RTX PRO 6000 Blackwell (sm120) - sglang patches, launchers, full optimization analysis☆30Aug 31, 2026Updated last month
- OpenATP is an open-source Python package providing a common interface for Automated Theorem Proving (ATP)☆30Oct 8, 2026Updated last week
- Pure C wrapper library to use llama.cpp with Linux and Windows as simple as possible.☆15Sep 6, 2026Updated last month
- 1-Click AI Models by DigitalOcean Gradient • AdDeploy popular AI models on DigitalOcean Gradient GPU virtual machines with just a single click. Zero configuration with optimized deployments.
- One backbone, many specialists. CMF is a self-describing, memory-mappable single-file format and a dependency-free runtime for quantized …☆33Updated this week
- Research / Toy Renderer☆11Jul 11, 2026Updated 3 months ago
- Mirror from gitlab☆11Jan 9, 2021Updated 5 years ago
- Qwen3.8-Flash-Next (MoE, NVFP4) and Qwen3.8-27B on two 16 GB consumer GPUs: two-GPU tensor parallel, experts offloaded to CPU. Formerly n…☆43Updated this week
- Pi coding agent extension for focused code retrieval, grounded edits, safety checks, and language-server support.☆28Sep 23, 2026Updated 3 weeks ago
- Publish mDNS records based on your Caddyfile☆15Jan 1, 2026Updated 9 months ago
- Personal pi agent setup☆16Mar 16, 2026Updated 7 months ago