Alternatives and similar repositories for RetrievalAttention

Users that are interested in RetrievalAttention are comparing it to the libraries listed below

• HugoZHL / PQCache
  [SIGMOD 2025] PQCache: Product Quantization-based KVCache for Long Context LLM Inference
  ☆81Updated 3 weeks ago
• YaoJiayi / CacheBlend
  ☆157Updated 5 months ago
• sspec-project / SparseSpec
  Accelerating Large-Scale Reasoning Model Inference with Sparse Self-Speculative Decoding
  ☆70Updated 3 weeks ago
• snu-comparch / InfiniGen
  InfiniGen: Efficient Generative Inference of Large Language Models with Dynamic KV Cache Management (OSDI'24)
  ☆168Updated last year
• LoongServe / LoongServe
  ☆126Updated last year
• hao-ai-lab / vllm-ltr
  [NeurIPS 2024] Efficient LLM Scheduling by Learning to Rank
  ☆66Updated last year
• microsoft / ParrotServe
  [OSDI'24] Serving LLM-based Applications Efficiently with Semantic Variable
  ☆204Updated last year
• hao-ai-lab / MuxServe
  ☆80Updated 2 months ago
• WukLab / preble
  Stateful LLM Serving
  ☆91Updated 9 months ago
• xinhao-luo / ClusterFusion
  [NeurIPS 2025] ClusterFusion: Expanding Operator Fusion Scope for LLM Inference via Cluster-Level Collective Primitive
  ☆51Updated 2 weeks ago
• pku-liang / ArkVale
  ArkVale: Efficient Generative LLM Inference with Recallable Key-Value Eviction (NIPS'24)
  ☆50Updated last year
• eddiegaoo / Apt-Serve
  ☆20Updated 6 months ago
• microsoft / chunk-attention
  ☆84Updated 8 months ago
• amazon-science / piperag
  PipeRAG: Fast Retrieval-Augmented Generation via Algorithm-System Co-design (KDD 2025)
  ☆29Updated last year
• alibaba-edu / qwen-bailian-usagetraces-anon
  ☆69Updated last month
• LLMServe / SwiftTransformer
  High performance Transformer implementation in C++.
  ☆146Updated 11 months ago
• alibaba / ServeGen
  A framework for generating realistic LLM serving workloads
  ☆93Updated 2 months ago
• tsinghua-ideal / Twilight
  [NeurIPS'25 Spotlight] Adaptive Attention Sparsity with Hierarchical Top-p Pruning
  ☆79Updated 3 weeks ago
• UChi-JCL / CacheGen
  ☆144Updated last year
• DerrickYLJ / TidalDecode
  [ICLR 2025] TidalDecode: A Fast and Accurate LLM Decoding with Position Persistent Sparse Attention
  ☆49Updated 4 months ago
• graphcore-research / llm-inference-research
  An experimentation platform for LLM inference optimisation
  ☆35Updated last year
• NEO-MLSys25 / NEO
  NEO is a LLM inference engine built to save the GPU memory crisis by CPU offloading
  ☆73Updated 6 months ago
• microsoft / tokenweave
  Efficient Compute-Communication Overlap for Distributed LLM Inference
  ☆66Updated last month
• andy-yang-1 / DoubleSparse
  16-fold memory access reduction with nearly no loss
  ☆109Updated 9 months ago
• Hanchenli / vllm-continuum
  Preview Code for Continuum Paper
  ☆19Updated 2 weeks ago
• CentML / Mist
  [EuroSys'25] Mist: Efficient Distributed Training of Large Language Models via Memory-Parallelism Co-Optimization
  ☆21Updated 4 months ago
• ruipeterpan / marconi
  Artifact for "Marconi: Prefix Caching for the Era of Hybrid LLMs" [MLSys '25 Outstanding Paper Award, Honorable Mention]
  ☆47Updated 9 months ago
• d-matrix-ai / keyformer-llm
  ☆58Updated last year
• gty111 / gLLM
  gLLM: Global Balanced Pipeline Parallelism System for Distributed LLM Serving with Token Throttling
  ☆51Updated last week
• timlee0212 / SiDA-MoE
  Code for MLSys 2024 Paper "SiDA-MoE: Sparsity-Inspired Data-Aware Serving for Efficient and Scalable Large Mixture-of-Experts Models"
  ☆22Updated last year