LLM KV cache compression made easy
β1,259Nov 16, 2026Updated this week
Alternatives and similar repositories for kvpress
Users that are interested in kvpress are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- π° Must-read papers on KV Cache Compression (constantly updating π€).β751Aug 16, 2026Updated 3 months ago
- β339Jul 10, 2025Updated last year
- [NeurIPS'25 Oral] Query-agnostic KV cache eviction: 3β4Γ reduction in memory and 2Γ decrease in latency (Qwen3/2.5, Gemma3, LLaMA3)β229Feb 11, 2026Updated 9 months ago
- [ICML 2024] KIVI: A Tuning-Free Asymmetric 2bit Quantization for KV Cacheβ441Nov 20, 2025Updated 11 months ago
- [NeurIPS'24 Spotlight, ICLR'25, ICML'25] To speed up Long-context LLMs' inference, approximate and dynamic sparse calculate the attentionβ¦β1,231Updated this week
- Simple, predictable pricing with DigitalOcean hosting β’ AdAlways know what you'll pay with monthly caps and flat pricing. Enterprise-grade infrastructure trusted by 600k+ customers.
- Unified KV Cache Compression Methods for Auto-Regressive Modelsβ1,394Aug 13, 2026Updated 3 months ago
- The Official Implementation of Ada-KV [NeurIPS 2025]β143Nov 26, 2025Updated 11 months ago
- [ICML 2024] Quest: Query-Aware Sparsity for Efficient Long-Context LLM Inferenceβ414Jul 10, 2025Updated last year
- [NeurIPS'23] H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models.β542Aug 1, 2024Updated 2 years ago
- KV cache compression for high-throughput LLM inferenceβ163Feb 5, 2025Updated last year
- Awesome-LLM-KV-Cache: A curated list of πAwesome LLM KV Cache Papers with Codes.β477Jun 17, 2026Updated 5 months ago
- FlashInfer: Kernel Library for LLM Servingβ6,958Updated this week
- [ICML 2025 Spotlight] ShadowKV: KV Cache in Shadows for High-Throughput Long-Context LLM Inferenceβ316May 1, 2025Updated last year
- This repository serves as a comprehensive survey of LLM development, featuring numerous research papers along with their corresponding coβ¦β374Jul 16, 2026Updated 4 months ago
- GPU virtual machines on DigitalOcean Gradient AI β’ AdGet to production fast with high-performance AMD and NVIDIA GPUs you can spin up in seconds. The definition of operational simplicity.
- [MLSys'25] QServe: W4A8KV4 Quantization and System Co-design for Efficient LLM Serving; [MLSys'25] LServe: Efficient Long-sequence LLM Seβ¦β870Mar 6, 2025Updated last year
- [ICLR2025 Spotlight] MagicPIG: LSH Sampling for Efficient LLM Generationβ256Dec 16, 2024Updated last year
- Accurate and fast KV cache compression with a gating mechanismβ35Jul 27, 2026Updated 3 months ago
- [ICLR 2025] DuoAttention: Efficient Long-Context LLM Inference with Retrieval and Streaming Headsβ545Feb 10, 2025Updated last year
- LMCache: Supercharge Your LLM with the Fastest KV Cache Layerβ12,398Updated this week
- Awesome LLM compression research papers and tools.β1,898Updated this week
- [ICLR 2025] Palu: Compressing KV-Cache with Low-Rank Projectionβ168Feb 20, 2025Updated last year
- π Efficient implementations for emerging model architecturesβ6,066Updated this week
- Official Implementation of EAGLE-1 (ICML'24), EAGLE-2 (EMNLP'24), and EAGLE-3 (NeurIPS'25).β2,581Feb 20, 2026Updated 9 months ago
- Deploy to Railway using AI coding agents - Free Credits Offer β’ AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- [COLM 2024] TriForce: Lossless Acceleration of Long Sequence Generation with Hierarchical Speculative Decodingβ280Aug 31, 2024Updated 2 years ago
- Train speculative decoding models effortlessly and port them smoothly to SGLang serving.β1,290Updated this week
- Efficient LLM Inference over Long Sequencesβ392Jun 25, 2025Updated last year
- Accelerating MoE with IO and Tile-aware Optimizationsβ791Aug 29, 2026Updated 2 months ago
- β47Nov 25, 2024Updated last year
- β39Updated this week
- FP16xINT4 LLM inference kernel that can achieve near-ideal ~4x speedups up to medium batchsizes of 16-32 tokens.β1,181Sep 4, 2024Updated 2 years ago
- NVIDIA Inference Xfer Library (NIXL)β1,371Updated this week
- Official Implementation for [ICLR26] DefensiveKV: Taming the Fragility of KV Cache Eviction in LLM Inferenceβ62Mar 28, 2026Updated 7 months ago
- 1-Click AI Models by DigitalOcean Gradient β’ AdDeploy popular AI models on DigitalOcean Gradient GPU virtual machines with just a single click. Zero configuration with optimized deployments.
- A library for accelerating Transformer models on NVIDIA GPUs, including using 8-bit and 4-bit floating point (FP8 and FP4) precision on Hβ¦β3,610Updated this week
- Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.β7,083Updated this week
- Distributed Compiler and Optimized Parallel Kernelsβ1,583Updated this week
- A sparse-first inference engine (SparseEngine).β110Nov 11, 2026Updated last week
- [ICML 2025] XAttention: Block Sparse Attention with Antidiagonal Scoringβ288Jul 6, 2025Updated last year
- Tile primitives for speedy kernelsβ3,803Sep 12, 2026Updated 2 months ago
- [NeurIPS 2024] KVQuant: Towards 10 Million Context Length LLM Inference with KV Cache Quantizationβ440Aug 13, 2024Updated 2 years ago
Are these results useful?