LINs-lab / DeFTLinks
[ICLR 2025] DeFT: Decoding with Flash Tree-attention for Efficient Tree-structured LLM Inference
β36Updated 3 months ago
Alternatives and similar repositories for DeFT
Users that are interested in DeFT are comparing it to the libraries listed below
Sorting:
- π₯ LLM-powered GPU kernel synthesis: Train models to convert PyTorch ops into optimized Triton kernels via SFT+RL. Multi-turn compilationβ¦β56Updated this week
- A Survey of Efficient Attention Methods: Hardware-efficient, Sparse, Compact, and Linear Attentionβ181Updated last month
- [NeurIPS 2024] Efficient LLM Scheduling by Learning to Rankβ59Updated 10 months ago
- Quantized Attention on GPUβ44Updated 10 months ago
- An efficient implementation of the NSA (Native Sparse Attention) kernelβ115Updated 3 months ago
- Accelerate LLM preference tuning via prefix sharing with a single line of codeβ43Updated 2 months ago
- Code for paper: [ICLR2025 Oral] FlexPrefill: A Context-Aware Sparse Attention Mechanism for Efficient Long-Sequence Inferenceβ142Updated 4 months ago
- β50Updated 4 months ago
- β38Updated last month
- 16-fold memory access reduction with nearly no lossβ105Updated 6 months ago
- Official Implementation of SAM-Decoding: Speculative Decoding via Suffix Automatonβ32Updated 7 months ago
- LongSpec: Long-Context Lossless Speculative Decoding with Efficient Drafting and Verificationβ63Updated 2 months ago
- β18Updated 9 months ago
- [ICLR 2025] TidalDecode: A Fast and Accurate LLM Decoding with Position Persistent Sparse Attentionβ47Updated last month
- [ICML 2025] SparseLoRA: Accelerating LLM Fine-Tuning with Contextual Sparsityβ56Updated 2 months ago
- Beyond KV Caching: Shared Attention for Efficient LLMsβ19Updated last year
- Pytorch implementation of our paper accepted by ICML 2024 -- CaM: Cache Merging for Memory-efficient LLMs Inferenceβ45Updated last year
- β58Updated 9 months ago
- Artifact for "Marconi: Prefix Caching for the Era of Hybrid LLMs" [MLSys '25 Outstanding Paper Award, Honorable Mention]β32Updated 6 months ago
- An auxiliary project analysis of the characteristics of KV in DiT Attention.β32Updated 9 months ago
- Kinetics: Rethinking Test-Time Scaling Lawsβ80Updated 2 months ago
- β87Updated 7 months ago
- Implementation for FP8/INT8 Rollout for RL training without performence drop.β207Updated last week
- SQUEEZED ATTENTION: Accelerating Long Prompt LLM Inferenceβ53Updated 10 months ago
- β43Updated 4 months ago
- β107Updated last month
- β126Updated 3 months ago
- β97Updated 4 months ago
- β50Updated last week
- β21Updated 6 months ago