sjtu-zhao-lab / ClusterKVLinks
ClusterKV: Manipulating LLM KV Cache in Semantic Space for Recallable Compression (DAC'25)
☆22Updated 4 months ago
Alternatives and similar repositories for ClusterKV
Users that are interested in ClusterKV are comparing it to the libraries listed below
Sorting:
- ArkVale: Efficient Generative LLM Inference with Recallable Key-Value Eviction (NIPS'24)☆52Updated last year
- InfiniGen: Efficient Generative Inference of Large Language Models with Dynamic KV Cache Management (OSDI'24)☆174Updated last year
- [NeurIPS'25 Spotlight] Adaptive Attention Sparsity with Hierarchical Top-p Pruning☆87Updated 2 months ago
- [NeurIPS 2025] ClusterFusion: Expanding Operator Fusion Scope for LLM Inference via Cluster-Level Collective Primitive☆66Updated last month
- Accelerating Large-Scale Reasoning Model Inference with Sparse Self-Speculative Decoding☆82Updated last month
- ☆130Updated last year
- [SIGMOD 2025] PQCache: Product Quantization-based KVCache for Long Context LLM Inference☆83Updated last month
- [HPCA 2026] A GPU-optimized system for efficient long-context LLMs decoding with low-bit KV cache.☆79Updated last month
- ☆164Updated 6 months ago
- Scalable long-context LLM decoding that leverages sparsity—by treating the KV cache as a vector storage system.☆119Updated last month
- The Official Implementation of Ada-KV [NeurIPS 2025]☆125Updated 2 months ago
- Keyformer proposes KV Cache reduction through key tokens identification and without the need for fine-tuning☆59Updated last year
- ☆83Updated last year
- This repository serves as a comprehensive survey of LLM development, featuring numerous research papers along with their corresponding co…☆271Updated last month
- ☆31Updated 9 months ago
- A tiny yet powerful LLM inference system tailored for researching purpose. vLLM-equivalent performance with only 2k lines of code (2% of …☆312Updated 7 months ago
- ☆15Updated last year
- Summary of some awesome work for optimizing LLM inference☆168Updated 2 months ago
- Paper reading and discussion notes, covering AI frameworks, distributed systems, cluster management, etc.☆53Updated 2 months ago
- Building the Virtuous Cycle for AI-driven LLM Systems☆140Updated this week
- ☆54Updated 4 months ago
- ☆58Updated last year
- ☆83Updated 3 months ago
- NEO is a LLM inference engine built to save the GPU memory crisis by CPU offloading☆79Updated 7 months ago
- SpInfer: Leveraging Low-Level Sparsity for Efficient Large Language Model Inference on GPUs☆61Updated 10 months ago
- Nex Venus Communication Library☆72Updated 2 months ago
- ☆143Updated last month
- High performance Transformer implementation in C++.☆148Updated last year
- Implement some method of LLM KV Cache Sparsity☆41Updated last year
- gLLM: Global Balanced Pipeline Parallelism System for Distributed LLM Serving with Token Throttling☆53Updated 2 weeks ago