Unified KV cache compression for LLM inference — TurboQuant, IsoQuant, PlanarQuant, TriAttention. 10 methods, GPU-validated, multi-GPU planner. Compress KV cache 5-80x to run bigger models, longer context, more agents on your GPU.
☆24Jul 11, 2026Updated last week
Alternatives and similar repositories for multi-turboquant
Users that are interested in multi-turboquant are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Specialized fork for (relatively) fast single-GPU inference (in CUDA) using large MoE models that don't fit fully into VRAM☆17May 6, 2026Updated 2 months ago
- LLM inference in C/C++☆64May 7, 2026Updated 2 months ago
- Object-storage-native KV cache for LLM inference & RL. Cross-restart, cross-conversation, cross-engine via shared S3 bucket.☆19Jul 6, 2026Updated 2 weeks ago
- LLM inference in C/C++☆36Apr 12, 2026Updated 3 months ago
- High-performance batched Top-K selection for CPU inference. Up to 80x faster than PyTorch, optimized for LLM sampling with AVX2 SIMD.☆17Mar 20, 2026Updated 4 months ago
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- Simple GUI editor for Ideogram V4's structured JSON prompts☆25Jun 10, 2026Updated last month
- TDS – Traffic Delivery System☆12Oct 9, 2021Updated 4 years ago
- High-performance K-line (Candlestick) chart for React Native, powered by Skia. Smooth, customizable, and built for real trading apps.☆19Mar 20, 2026Updated 4 months ago
- DFlash vLLM for DGX Spark — Plug & Play Block-Diffusion Speculative Decoding☆52Jun 28, 2026Updated 3 weeks ago
- RWKV-7 mini☆12Mar 29, 2025Updated last year
- A2A MCP Server is a lightweight Python bridge that lets Claude Desktop or any MCP client talk to A2A agents. It provides three tools: reg…☆21May 4, 2025Updated last year
- JavaScript component to parse, clean, remove formatting (unformat) numbers in strings.☆10Dec 5, 2024Updated last year
- Dispatch AI coding agents (Claude Code, Copilot, OpenCode) as MCP tools☆21Apr 8, 2026Updated 3 months ago
- ☆12Dec 8, 2022Updated 3 years ago
- Bare Metal GPUs on DigitalOcean Gradient AI • AdPurpose-built for serious AI teams training foundational models, running large-scale inference, and pushing the boundaries of what's possible.
- Manages tasks and persistent storage for AI agents — assigns work, tracks completion, and provides a shared key-value store with full-tex…☆32Updated this week
- Tiny evaluation of leading LLMs on competitive programming problems☆14Apr 10, 2026Updated 3 months ago
- TurboDB — a blazing-fast NoSQL document database written in Zig. mmap + WAL + B-tree + MVCC.☆19May 1, 2026Updated 2 months ago
- A simple NER implementation using a DistilBERT based model with ML.NET☆13May 6, 2021Updated 5 years ago
- Towards Robust Fact-Checking: A Multi-Agent System with Advanced Evidence Retrieval☆15Jun 24, 2025Updated last year
- This includes 2 separate tutorial series for OpenAI swarm library each 10 files from basic to advanced☆14Jan 14, 2025Updated last year
- Project used to generate ML.NET AutoML code for machine learning.☆11Jul 19, 2021Updated 5 years ago
- SyzgyDB: An embeddable vector database in Go for efficient disk-based storage and similarity searches, supporting various distance metric…☆11Nov 1, 2024Updated last year
- Extract structured data from documents, images, audio, and video using LLMs.☆18Updated this week
- GPU virtual machines on DigitalOcean Gradient AI • AdGet to production fast with high-performance AMD and NVIDIA GPUs you can spin up in seconds. The definition of operational simplicity.
- TurboQuant KV Cache Compression for llama.cpp — 5.2x memory reduction with near-lossless quality | Implementation of Google DeepMind's …☆91Jul 12, 2026Updated last week
- AGI runtime for bounded recursive self-awareness. Attempting machine consciousness at the Gödel–Turing–Hofstadter Nexus.☆16Mar 24, 2026Updated 3 months ago
- Elasticsearch provider for Examine in Umbraco v8☆12Jan 15, 2024Updated 2 years ago
- MCP server for Semantic Scholar to search for papers☆20Apr 5, 2025Updated last year
- small board to convert linear 3 pin regulator to switching regulator (TO220 compatible, eg 7805)☆11Oct 8, 2015Updated 10 years ago
- Zig implementation of Apache Arrow☆15Jun 14, 2026Updated last month
- Qwen LLM in the mac menu bar <3☆27Mar 12, 2025Updated last year
- Assembler written in C# for 6502 based systems☆10Dec 17, 2022Updated 3 years ago
- The purpose of these documents, it was to execute several efficiency and detection tests in some endpoint solutions, this document brings…☆21Dec 23, 2020Updated 5 years ago
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- ☆20Feb 2, 2025Updated last year
- CNTK implementation of Fully Convolutional Networks (FCN) with ResNet for semantic segmentation☆12Aug 18, 2017Updated 8 years ago
- ☆15Feb 15, 2017Updated 9 years ago
- ☆389Apr 16, 2026Updated 3 months ago
- NVFP4 Gemma-4 26B-A4B MoE for DGX Spark — optimal recipe: DFlash n=10 (flex) on AEON vLLM Ultimate. 144 tok/s single / 1,724 peak (Coding…☆42Jun 28, 2026Updated 3 weeks ago
- A lightweight, embeddable SQL parser designed for SQLite-based applications. It parses SQL statements into a structured representation th…☆22Apr 28, 2026Updated 2 months ago
- Semantic Kernel connector for ONNX models.☆12Jun 10, 2024Updated 2 years ago