llama.cpp fork with TurboQuant WHT-rotated KV cache & weight compression + Gemma 4 MTP and Qwen 3.6 NextN speculative decoding (+30-50% throughput).
☆435Nov 10, 2026Updated last week
Alternatives and similar repositories for atomic-llama-cpp-turboquant
Users that are interested in atomic-llama-cpp-turboquant are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- LLM inference in C/C++☆2,454Nov 13, 2026Updated last week
- KVarN, KV cache precision tail, low-bit quants in llama.cpp for longer context of better precision in the same VRAM☆1,199Updated this week
- Qwen4-Exp (Qwen3.8-Flash-Next) SWA + MTP + PLE-on-iswa — bounded deep-context decode w/ long-range recall, TBQ4 4.25 bpv KV, DSpark + Rot…☆94Sep 24, 2026Updated last month
- llama.cpp fork with TQ3_1S/4S CUDA kernels — 3.5-bit WHT quantization achieving Q4s quality at 10% smaller size. Based on RaBitQ-inspired…☆237Nov 5, 2026Updated 2 weeks ago
- Experimental llama.cpp fork for inference research and development☆910Nov 13, 2026Updated last week
- Virtual machines for every use case on DigitalOcean • AdGet dependable uptime with 99.99% SLA, simple security tools, and predictable monthly pricing with DigitalOcean's virtual machines, called Droplets.
- ☆148Jun 13, 2026Updated 5 months ago
- NInfer-KVMem-Async☆12Updated this week
- llama.cpp fork with additional SOTA quants and improved performance☆3,408Updated this week
- LLM inference in C/C++☆1,026Updated this week
- Local AI app and inference engine for agents. Run open-weight LLMs locally — private, 100% offline on your computer. Join our Discord: ht…☆2,034Updated this week
- LLM inference in C/C++☆65May 7, 2026Updated 6 months ago
- Lucebox inference engine for heterogeneous hardware & consumer GPUs☆3,023Updated this week
- Adaptive Precision for EXpert Models: MoE-aware mixed-precision quantization☆489Oct 14, 2026Updated last month
- The Fastest Way to Run OpenClaw 🦞☆327May 26, 2026Updated 5 months ago
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- KV cache compression via block-diagonal rotation. Beats TurboQuant: better PPL (6.91 vs 7.07), 28% faster decode, 5.3x faster prefill, 44…☆1,059Apr 23, 2026Updated 6 months ago
- Fermion Research fork of llama.cpp: FV5/FV5B five-value ternary types for Neutrino GGUF packs (CPU + CUDA)☆14Sep 29, 2026Updated last month
- LLM inference in C/C++☆488Updated this week
- KVMem virtualizes KV memory across inference backends to make ultra-long-context local AI accessible on limited hardware.☆1,365Updated this week
- LLM inference in C/C++☆15Aug 27, 2026Updated 2 months ago
- TurboQuant llama.cpp fork with optimized turbo4 kernels for Gemma 4 D=256/512 heads — lazy K/V, batch decode, warp-cooperative write. 120…☆37Apr 5, 2026Updated 7 months ago
- Atomic Chat terminal workspace with local chat, model setup and an API server console☆47Updated this week
- Qwen3.8-Flash-Next on any consumer hardware: one-click install for Windows / Linux. Strata inference engine, OpenAI/Anthropic API on loca…☆46,212Updated this week
- Reliable model swapping for any local AI server. Chosen by thousands of users. Batteries included. If we don't have it, you probably don…☆6,281Updated this week
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- An all-in-one, pure C++ inference engine for audio models, powered by ggml. Supports TTS, STT, VAD, voice conversion, music generation, a…☆4,372Updated this week
- LLM inference in C/C++☆55Updated this week
- Durable local inference for Oh My Pi: NInfer + Qwen3.8 27B on one RTX 3090/4090/5090, with restart-resumable OpenAI Responses state.☆51Updated this week
- Run large AI models on your own PC: Qwen3.8-Flash-Next on Strata's engine, and GLM 5.3 Flash, DeepSeek V4 Flash, LFM2.5, Qwen3.6, Ornith …☆1,296Updated this week
- 🐾 Complete REST API for ClawHub skills marketplace data☆16Apr 6, 2026Updated 7 months ago
- DFlash: Block Diffusion for Flash Speculative Decoding☆6,254Aug 18, 2026Updated 3 months ago
- Experimental LLM inference in C/C++☆38May 15, 2026Updated 6 months ago
- Run GLM-5.3 (744B), DeepSeek-V4.1-Flash (748B), MiniMax-M3.1-Flash (428B), GLM-5.3-Flash (321B) and Hy4-Flash (220B) on your own GPUs - f…☆3,106Updated this week
- Bonsai Demo☆3,536Nov 5, 2026Updated 2 weeks ago
- Proton VPN Special Offer - Get 70% off • AdSpecial partner offer. Trusted by over 100 million users worldwide. Tested, Approved and Recommended by Experts.
- Gradient-free weight editing reduces overthinking in Qwen3.8-27B — analysis code and evaluation artifacts☆36Nov 8, 2026Updated last week
- Run Qwen Flash Next on a Mac Mini m5 64GB box☆20Updated this week
- High-performance single-GPU inference for selected model checkpoints and GPUs.☆3,387Updated this week
- Produce your own Dynamic 3.0 Quants and achieve optimum accuracy & SOTA quantization performance! Input a target size and the toolchain w…☆174Sep 11, 2026Updated 2 months ago
- llama.cpp fork with TurboQuant quantization (turbo2/3/4) and TriAttention GPU-accelerated KV cache pruning. 75 tok/s on Qwen3-8B / RTX 30…☆59Jul 2, 2026Updated 4 months ago
- User friendly GUI for configuring and launching llama.cpp☆145Updated this week
- Smart OpenAI‑compatible proxy for llama.cpp: manages slots, saves/restores KV cache to disk, routes requests by prefix similarity, and pr…☆57Sep 16, 2026Updated 2 months ago
Are these results useful?