Alternatives and similar repositories for flash_tree_attn:

Users that are interested in flash_tree_attn are comparing it to the libraries listed below

• SqueezeAILab / SqueezedAttention
  SQUEEZED ATTENTION: Accelerating Long Prompt LLM Inference
  ☆42Updated 3 months ago
• Dominic789654 / LongGenBench
  Source code for the paper "LongGenBench: Long-context Generation Benchmark"
  ☆14Updated 4 months ago
• hyx1999 / SAM-Decoding
  Official Implementation of SAM-Decoding: Speculative Decoding via Suffix Automaton
  ☆19Updated 2 weeks ago
• LiuXiaoxuanPKU / OSD
  ☆41Updated 3 months ago
• TiledTensor / TiledBench
  Benchmark tests supporting the TiledCUDA library.
  ☆15Updated 3 months ago
• smart-lty / ParallelSpeculativeDecoding
  [ICLR 2025] PEARL: parallel speculative decoding with adaptive draft length
  ☆44Updated this week
• DerrickYLJ / TidalDecode
  [ICLR 2025] TidalDecode: A Fast and Accurate LLM Decoding with Position Persistent Sparse Attention
  ☆29Updated this week
• sail-sg / SimLayerKV
  The official implementation of paper: SimLayerKV: A Simple Framework for Layer-Level KV Cache Reduction.
  ☆42Updated 4 months ago
• feifeibear / ChituAttention
  Quantized Attention on GPU
  ☆34Updated 3 months ago
• FFY0 / AdaKV
  The Official Implementation of Ada-KV: Optimizing KV Cache Eviction by Adaptive Budget Allocation for Efficient LLM Inference
  ☆63Updated last month
• BBuf / flash-rwkv
  ☆30Updated 9 months ago
• zyxxmu / cam
  Pytorch implementation of our paper accepted by ICML 2024 -- CaM: Cache Merging for Memory-efficient LLMs Inference
  ☆33Updated 8 months ago
• feifeibear / DPSKV3MFU
  Estimate MFU for DeepSeekV3
  ☆18Updated last month
• NJUNLP / MCSD
  Multi-Candidate Speculative Decoding
  ☆34Updated 10 months ago
• dame-cell / Triformer
  Transformers components but in Triton
  ☆31Updated 3 months ago
• hemingkx / SWIFT
  [ICLR 2025] SWIFT: On-the-Fly Self-Speculative Decoding for LLM Inference Acceleration
  ☆38Updated last week
• Zanette-Labs / SpeculativeRejection
  [NeurIPS 2024] Fast Best-of-N Decoding via Speculative Rejection
  ☆39Updated 4 months ago
• VITA-Group / Ms-PoE
  "Found in the Middle: How Language Models Use Long Contexts Better via Plug-and-Play Positional Encoding" Zhenyu Zhang, Runjin Chen, Shiw…
  ☆26Updated 9 months ago
• hdong920 / LESS
  ☆49Updated 9 months ago
• BaiTheBest / SparseLLM
  Official Repo for SparseLLM: Global Pruning of LLMs (NeurIPS 2024)
  ☆52Updated 3 weeks ago
• Equationliu / Kangaroo
  [NeurIPS 2024] The official implementation of "Kangaroo: Lossless Self-Speculative Decoding for Accelerating LLMs via Double Early Exitin…
  ☆49Updated 8 months ago
• Jikai0Wang / OPT-Tree
  OPT-Tree: Speculative Decoding with Adaptive Draft Tree Structure
  ☆21Updated 6 months ago
• Aleph-Alpha / NeurIPS-WANT-submission-efficient-parallelization-layouts
  ☆22Updated last year
• raymin0223 / fast_robust_early_exit
  Fast and Robust Early-Exiting Framework for Autoregressive Language Models with Synchronized Parallel Decoding (EMNLP 2023 Long)
  ☆56Updated 5 months ago
• microsoft / chunk-attention
  ☆62Updated 2 months ago