llama.cpp fork with TurboQuant quantization (turbo2/3/4) and TriAttention GPU-accelerated KV cache pruning. 75 tok/s on Qwen3-8B / RTX 3080.
☆54Jul 2, 2026Updated 2 months ago
Alternatives and similar repositories for llama-cpp-turboquant
Users that are interested in llama-cpp-turboquant are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- LLM inference in C/C++☆2,386Updated this week
- Unified KV-cache compression for LLM inference: 12 Python-native methods, guarded add-on composition and routing, analytical capacity sim…☆26Aug 22, 2026Updated 3 weeks ago
- LLM inference in C/C++☆64May 7, 2026Updated 4 months ago
- Road System is unity package to procedural roads generation.☆12Nov 10, 2024Updated last year
- ☆154Jun 13, 2026Updated 3 months ago
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- Community-maintained fork of the classic Python Open Interpreter. The classic/develop branch is what I use on a regular basis, and it ha…☆33Updated this week
- AutoMapper website☆14Aug 19, 2020Updated 6 years ago
- Simple MCP server for uithub.com☆21Nov 5, 2025Updated 10 months ago
- Calculate discrete frechet distance.☆11Feb 21, 2016Updated 10 years ago
- Implementaion of "A Simple Approach to Recognise Geometric Shapes Interactively" by Joaquim A. Jorge and Manuel J. Fonseca.☆19Jan 7, 2023Updated 3 years ago
- See what is in the dark fully functioning in the browser.☆22Jan 29, 2024Updated 2 years ago
- 基于.net core2.1 搭建的框架的中小型项目框架,sql server数据库,同时支持,mysql数据库,,,需要自己稍微扩展下,,,可以擦模考sql server的实现 迁移文件已经生成,可以直接运行即可跑起来. 结合docker发布可以参考这里: https…☆13Dec 4, 2018Updated 7 years ago
- 开源网址导航☆13Apr 15, 2021Updated 5 years ago
- KVarN, KV cache precision tail, low-bit quants in llama.cpp for longer context of better precision in the same VRAM☆1,096Sep 13, 2026Updated last week
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- ☆91Mar 10, 2026Updated 6 months ago
- Unofficial demo app for CogView2☆16Jul 30, 2022Updated 4 years ago
- ☆10Aug 10, 2015Updated 11 years ago
- llama.cpp fork with TurboQuant WHT-rotated KV cache & weight compression + Gemma 4 MTP and Qwen 3.6 NextN speculative decoding (+30-50% t…☆411Sep 10, 2026Updated last week
- Demo for JWT☆30Aug 30, 2019Updated 7 years ago
- RedisMQDemo☆11Jun 13, 2019Updated 7 years ago
- Implementations of UUID v4 and v7 as defined in the lastest RFC4122 draft. Including a highly-performant custom UUID v8 implementation.☆12May 27, 2025Updated last year
- A Python package designed to simplify the process of creating and managing function calls to OpenAI's API, as well as models using LiteLL…☆17May 25, 2025Updated last year
- Docker container for use with Multi-layer Recurrent Neural Networks (LSTM, GRU, RNN) for character-level language models in Torch☆10Jan 19, 2019Updated 7 years ago
- AI Agents on DigitalOcean Gradient AI Platform • AdBuild production-ready AI agents using customizable tools or access multiple LLMs through a single endpoint. Create custom knowledge bases or connect external data.
- 💆♂️ Hassle-free export your palette to SVG format☆16Jan 4, 2023Updated 3 years ago
- AceForge is a local-first AI music workstation for Apple/OSX based on Ace-Step, DeMucs, XTTSv2☆92Feb 11, 2026Updated 7 months ago
- Image Text Segmentation using FAST corner detection and DBSCAN clustering with k-d tree data structure☆14Feb 27, 2019Updated 7 years ago
- [AAAI'23] FinalMLP: An Enhanced Two-Stream MLP Model for CTR Prediction https://arxiv.org/abs/2304.00902☆11Apr 9, 2023Updated 3 years ago
- A gRPC-Web implementation for Python☆14Jan 28, 2025Updated last year
- ☆13Mar 13, 2023Updated 3 years ago
- ISPyB☆13Jul 16, 2026Updated 2 months ago
- .NET 跨平台微服务架构解决方案☆16Jul 10, 2026Updated 2 months ago
- ASP.NET Core WebApi EntityFrameworkCore 连接mysql 、sqlserver的样例工程☆10Jan 4, 2017Updated 9 years ago
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- Patched native-Windows build of vLLM. Three Windows-specific fixes (CPU-relay for Gloo, Qwen3 reasoning parser, wildcard model name) on t…☆28May 8, 2026Updated 4 months ago
- Man machine verification module based on ABP vNext☆14Aug 11, 2025Updated last year
- A minimalistic ASP.NET 4.5.1/MVC 5/EF 6/Bootstrap 3.1.1 based content management system☆10Sep 21, 2015Updated 10 years ago
- A library for queueing tasks in Redis☆15Mar 10, 2016Updated 10 years ago
- FFMPEG.js Examples in browser☆16Apr 16, 2019Updated 7 years ago
- TurboQuant llama.cpp fork with optimized turbo4 kernels for Gemma 4 D=256/512 heads — lazy K/V, batch decode, warp-cooperative write. 120…☆36Apr 5, 2026Updated 5 months ago
- Stackdriver Logging / GCP Cloud Logging handler for the go 1.21 slog package.☆11Feb 24, 2025Updated last year