NoakLiu / PiKVLinks
PiKV: KV Cache Management System for Mixture of Experts [Efficient ML System]
☆48Updated 3 months ago
Alternatives and similar repositories for PiKV
Users that are interested in PiKV are comparing it to the libraries listed below
Sorting:
- [DAC'25] Official implement of "HybriMoE: Hybrid CPU-GPU Scheduling and Cache Management for Efficient MoE Inference"☆101Updated last month
- ☆52Updated 8 months ago
- Accelerating Large-Scale Reasoning Model Inference with Sparse Self-Speculative Decoding☆87Updated 2 months ago
- 16-fold memory access reduction with nearly no loss☆110Updated 10 months ago
- [ICLR 2025] TidalDecode: A Fast and Accurate LLM Decoding with Position Persistent Sparse Attention☆53Updated 6 months ago
- Code for paper: [ICLR2025 Oral] FlexPrefill: A Context-Aware Sparse Attention Mechanism for Efficient Long-Sequence Inference☆160Updated 3 months ago
- ☆85Updated 3 months ago
- Quantized Attention on GPU☆44Updated last year
- [ICLR2025] Breaking Throughput-Latency Trade-off for Long Sequences with Speculative Decoding☆141Updated last year
- Distributed MoE in a Single Kernel [NeurIPS '25]☆191Updated this week
- DeeperGEMM: crazy optimized version☆73Updated 9 months ago
- Accepted to MLSys 2026☆70Updated last week
- GEAR: An Efficient KV Cache Compression Recipefor Near-Lossless Generative Inference of LLM☆176Updated last year
- Compression for Foundation Models☆35Updated 6 months ago
- Artifact for "Marconi: Prefix Caching for the Era of Hybrid LLMs" [MLSys '25 Outstanding Paper Award, Honorable Mention]☆49Updated 11 months ago
- ☆65Updated 9 months ago
- [ACL 2025] Squeezed Attention: Accelerating Long Prompt LLM Inference☆56Updated last year
- Efficient Long-context Language Model Training by Core Attention Disaggregation☆87Updated last week
- PipeRAG: Fast Retrieval-Augmented Generation via Algorithm-System Co-design (KDD 2025)☆30Updated last year
- [NeurIPS'25 Spotlight] Adaptive Attention Sparsity with Hierarchical Top-p Pruning☆87Updated 2 months ago
- High-performance distributed data shuffling (all-to-all) library for MoE training and inference☆112Updated last month
- ☆22Updated 11 months ago
- AdaSkip: Adaptive Sublayer Skipping for Accelerating Long-Context LLM Inference☆20Updated last year
- Autonomous GPU Kernel Generation via Deep Agents☆228Updated this week
- Decoding Attention is specially optimized for MHA, MQA, GQA and MLA using CUDA core for the decoding stage of LLM inference.☆46Updated 8 months ago
- ☆130Updated 5 months ago
- [ICLR'25] Fast Inference of MoE Models with CPU-GPU Orchestration☆260Updated last year
- Explore Inter-layer Expert Affinity in MoE Model Inference☆16Updated last year
- An experimental communicating attention kernel based on DeepEP.☆35Updated 6 months ago
- ☆118Updated 8 months ago