llama.cpp fork with TurboQuant quantization (turbo2/3/4) and TriAttention GPU-accelerated KV cache pruning. 75 tok/s on Qwen3-8B / RTX 3080.
☆41Jul 2, 2026Updated last month
Alternatives and similar repositories for llama-cpp-turboquant
Users that are interested in llama-cpp-turboquant are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- LLM inference in C/C++☆2,234Updated this week
- Unified KV-cache compression for LLM inference: 12 Python-native methods, Debian-tested isolated add-ons, Godzilla KVarN/TriAttention, ex…☆25Updated this week
- LLM inference in C/C++☆64May 7, 2026Updated 2 months ago
- ☆146Jun 13, 2026Updated last month
- vLLM fork for Tesla V100 (SM70) — extends 1CatAI's AWQ support and adds GGUF support☆20Jun 20, 2026Updated last month
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- Community-maintained fork of the classic Python Open Interpreter. The classic/develop branch is what I use on a regular basis, and it ha…☆30Updated this week
- chrome extension☆18Jun 29, 2020Updated 6 years ago
- A common protocol for AI agent tools☆10Oct 21, 2024Updated last year
- KVarN, KV cache precision tail, low-bit quants in llama.cpp for longer context of better precision in the same VRAM☆833Updated this week
- CamanJS filter to white balance a canvas with a RGB color or color temperature in Kelvin☆11Jul 29, 2016Updated 10 years ago
- 开源网址导航☆13Apr 15, 2021Updated 5 years ago
- An experiment with movie scenes and contrastive learning☆11Feb 1, 2025Updated last year
- ☆10Aug 10, 2015Updated 10 years ago
- Memo's Blog☆27Jan 21, 2026Updated 6 months ago
- GPU virtual machines on DigitalOcean Gradient AI • AdGet to production fast with high-performance AMD and NVIDIA GPUs you can spin up in seconds. The definition of operational simplicity.
- MacOS dragging helper☆12Mar 31, 2024Updated 2 years ago
- CSS3与页面布局学习总结（一）——概要、选择器、特殊性与刻度单位☆14Nov 24, 2016Updated 9 years ago
- llama.cpp fork with TurboQuant WHT-rotated KV cache & weight compression + Gemma 4 MTP and Qwen 3.6 NextN speculative decoding (+30-50% t…☆325Updated this week
- Samples on how to use filters to authenticate ASP .Net Web Api 2☆10Oct 4, 2016Updated 9 years ago
- 一些 CorelDRAW Graphics Suite 常用插件备份☆16Mar 9, 2025Updated last year
- Docker container for use with Multi-layer Recurrent Neural Networks (LSTM, GRU, RNN) for character-level language models in Torch☆10Jan 19, 2019Updated 7 years ago
- AceForge is a local-first AI music workstation for Apple/OSX based on Ace-Step, DeMucs, XTTSv2☆85Feb 11, 2026Updated 5 months ago
- TurboDB — a blazing-fast NoSQL document database written in Zig. mmap + WAL + B-tree + MVCC.☆20May 1, 2026Updated 3 months ago
- A gRPC-Web implementation for Python☆14Jan 28, 2025Updated last year
- Wordpress hosting with auto-scaling - Free Trial Offer • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- ☆13Mar 13, 2023Updated 3 years ago
- .NET 跨平台微服务架构解决方案☆16Jul 10, 2026Updated 3 weeks ago
- 基于 FastAPI 的 DeepSeek Chat 反向代理，将 DeepSeek 网页版的 API 转换为 OpenAI 兼容格式。 支持流式/非流式对话、专家模式、深度思考（reasoning_content）、工具调用（DSML prompt injectio…☆60Updated this week
- Whisper in combination with GPT-3☆16Oct 13, 2022Updated 3 years ago
- ASP.NET Core WebApi EntityFrameworkCore 连接mysql 、sqlserver的样例工程☆10Jan 4, 2017Updated 9 years ago
- Patched native-Windows build of vLLM. Three Windows-specific fixes (CPU-relay for Gloo, Qwen3 reasoning parser, wildcard model name) on t…☆25May 8, 2026Updated 2 months ago
- Provides safety features used by the horde.☆15Jul 22, 2025Updated last year
- Back-End desenvolvido com C# utilizando .NET Core + ASP.NET Core WebApi + DDD (Domain Drive Design) + SOLID + CQRS + Design Patterns☆13Sep 6, 2018Updated 7 years ago
- A library for queueing tasks in Redis☆15Mar 10, 2016Updated 10 years ago
- Open source password manager - Proton Pass • AdSecurely store, share, and autofill your credentials with Proton Pass, the end-to-end encrypted password manager trusted by millions.
- FFMPEG.js Examples in browser☆16Apr 16, 2019Updated 7 years ago
- TurboQuant llama.cpp fork with optimized turbo4 kernels for Gemma 4 D=256/512 heads — lazy K/V, batch decode, warp-cooperative write. 120…☆35Apr 5, 2026Updated 4 months ago
- Modal LLM LLama.cpp based model deployment as part of series of Model as a Service (MaaS)☆17Mar 23, 2026Updated 4 months ago
- Stackdriver Logging / GCP Cloud Logging handler for the go 1.21 slog package.☆10Feb 24, 2025Updated last year
- Zig implementation of Apache Arrow☆15Jun 14, 2026Updated last month
- Docker version API for MODNet-model Human Matting☆16Dec 3, 2023Updated 2 years ago
- 让零基础小白也能一键安装运行 OpenClaw AI 编程助手，无需配置环境、无需命令行 | Tauri + React + Rust☆19Apr 10, 2026Updated 3 months ago