llama.cpp fork with TurboQuant quantization (turbo2/3/4) and TriAttention GPU-accelerated KV cache pruning. 75 tok/s on Qwen3-8B / RTX 3080.
☆55Jul 2, 2026Updated 3 months ago
Alternatives and similar repositories for llama-cpp-turboquant
Users that are interested in llama-cpp-turboquant are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- LLM inference in C/C++☆2,423Updated this week
- Unified KV-cache compression for LLM inference: 12 Python-native methods, guarded add-on composition and routing, analytical capacity sim…☆28Aug 22, 2026Updated last month
- LLM inference in C/C++☆64May 7, 2026Updated 5 months ago
- ☆151Jun 13, 2026Updated 3 months ago
- vLLM fork for Tesla V100 (SM70) — extends 1CatAI's AWQ support and adds GGUF support☆24Jun 20, 2026Updated 3 months ago
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- Community-maintained fork of the classic Python Open Interpreter. The classic/develop branch is what I use on a regular basis, and it ha…☆33Updated this week
- common go code☆14Feb 22, 2021Updated 5 years ago
- Simple MCP server for uithub.com☆21Nov 5, 2025Updated 11 months ago
- 🍻 Homebrew formula for Meetily backend : A CLI workflow for the administration of macOS backend applications distributed as binaries. ht…☆26Oct 23, 2025Updated 11 months ago
- Fix broken dev servers instantly. A fast Rust CLI for port conflicts, zombie processes, and dev server recovery.☆15Apr 8, 2026Updated 6 months ago
- A common protocol for AI agent tools☆10Oct 21, 2024Updated last year
- New Construction Options Mod for C&C Remastered - Tiberian Dawn☆12Dec 8, 2022Updated 3 years ago
- KVarN, KV cache precision tail, low-bit quants in llama.cpp for longer context of better precision in the same VRAM☆1,150Updated this week
- ☆92Mar 10, 2026Updated 7 months ago
- Simple, predictable pricing with DigitalOcean hosting • AdAlways know what you'll pay with monthly caps and flat pricing. Enterprise-grade infrastructure trusted by 600k+ customers.
- MacOS dragging helper☆12Mar 31, 2024Updated 2 years ago
- A Python Program Based On BeautifulSoup To Search The User's Desired Movies On https://yts.mx And Return Movies And Torrent Info☆10Mar 24, 2023Updated 3 years ago
- llama.cpp fork with TurboQuant WHT-rotated KV cache & weight compression + Gemma 4 MTP and Qwen 3.6 NextN speculative decoding (+30-50% t…☆420Updated this week
- Implementations of UUID v4 and v7 as defined in the lastest RFC4122 draft. Including a highly-performant custom UUID v8 implementation.☆12May 27, 2025Updated last year
- A mod that allows you to add/replace existing files without repacking CRI Middleware CPKs.☆15Jun 14, 2026Updated 3 months ago
- ☆11Nov 29, 2023Updated 2 years ago
- Docker container for use with Multi-layer Recurrent Neural Networks (LSTM, GRU, RNN) for character-level language models in Torch☆10Jan 19, 2019Updated 7 years ago
- Ngram for golang☆17Feb 20, 2018Updated 8 years ago
- AceForge is a local-first AI music workstation for Apple/OSX based on Ace-Step, DeMucs, XTTSv2☆94Feb 11, 2026Updated 7 months ago
- AI Agents on DigitalOcean Gradient AI Platform • AdBuild production-ready AI agents using customizable tools or access multiple LLMs through a single endpoint. Create custom knowledge bases or connect external data.
- ☆17Jan 2, 2024Updated 2 years ago
- ☆13Mar 13, 2023Updated 3 years ago
- 基于Winform的一套商家管理工具，用来练习界面布局，自动更新，事件，异步，多线程，网络请求，缓存使用，任务存储，设计模式，爬虫流程，反爬虫机制等功能的一个Demo☆10Dec 8, 2022Updated 3 years ago
- Golang SDK for Google actions☆11Jun 29, 2018Updated 8 years ago
- Patched native-Windows build of vLLM. Three Windows-specific fixes (CPU-relay for Gloo, Qwen3 reasoning parser, wildcard model name) on t…☆28May 8, 2026Updated 5 months ago
- ASP.NET Core WebApi EntityFrameworkCore 连接mysql 、sqlserver的样例工程☆10Jan 4, 2017Updated 9 years ago
- Provides safety features used by the horde.☆15Aug 25, 2026Updated last month
- A library for queueing tasks in Redis☆15Mar 10, 2016Updated 10 years ago
- ☆17Jun 14, 2018Updated 8 years ago
- Deploy open-source AI quickly and easily - Special Bonus Offer • AdRunpod Hub is built for open source. One-click deployment and autoscaling endpoints without provisioning your own infrastructure.
- Modal LLM LLama.cpp based model deployment as part of series of Model as a Service (MaaS)☆17Mar 23, 2026Updated 6 months ago
- TurboQuant llama.cpp fork with optimized turbo4 kernels for Gemma 4 D=256/512 heads — lazy K/V, batch decode, warp-cooperative write. 120…☆37Apr 5, 2026Updated 6 months ago
- GitHub Action that leverages Google´s Container Structure test to run unit tests on your container images.☆10Sep 6, 2026Updated last month
- ☆13Jan 8, 2024Updated 2 years ago
- Zig implementation of Apache Arrow☆19Jun 14, 2026Updated 3 months ago
- 使用Swagger 搭建 WebApi文档项目☆10Aug 19, 2018Updated 8 years ago
- A custom CSS theme for Obsidian inspired by The Incredible Hulk☆18Jul 13, 2021Updated 5 years ago