epfml / dynamic-sparse-flash-attentionLinks
☆144Updated 2 years ago
Alternatives and similar repositories for dynamic-sparse-flash-attention
Users that are interested in dynamic-sparse-flash-attention are comparing it to the libraries listed below
Sorting:
- Code for exploring Based models from "Simple linear attention language models balance the recall-throughput tradeoff"☆233Updated 3 months ago
- Triton-based implementation of Sparse Mixture of Experts.☆216Updated 6 months ago
- Some preliminary explorations of Mamba's context scaling.☆212Updated last year
- ☆108Updated last year
- Fast and memory-efficient exact attention☆68Updated 2 months ago
- Official repository for DistFlashAttn: Distributed Memory-efficient Attention for Long-context LLMs Training☆208Updated 9 months ago
- Understand and test language model architectures on synthetic tasks.☆195Updated 2 months ago
- A MAD laboratory to improve AI architecture designs 🧪☆116Updated 5 months ago
- ☆81Updated last year
- 🔥 A minimal training framework for scaling FLA models☆141Updated 2 weeks ago
- Repository of the paper "Accelerating Transformer Inference for Translation via Parallel Decoding"☆116Updated last year
- This repository contains the experimental PyTorch native float8 training UX☆222Updated 10 months ago
- Tree Attention: Topology-aware Decoding for Long-Context Attention on GPU clusters☆126Updated 5 months ago
- Triton implementation of FlashAttention2 that adds Custom Masks.☆117Updated 9 months ago
- [NeurIPS'23] Speculative Decoding with Big Little Decoder☆92Updated last year
- Explorations into some recent techniques surrounding speculative decoding☆266Updated 5 months ago
- ☆103Updated last year
- ☆105Updated 9 months ago
- GEAR: An Efficient KV Cache Compression Recipefor Near-Lossless Generative Inference of LLM☆163Updated 10 months ago
- Low-bit optimizers for PyTorch☆128Updated last year
- Repo for "LoLCATs: On Low-Rank Linearizing of Large Language Models"☆237Updated 4 months ago
- Accelerated First Order Parallel Associative Scan☆180Updated 9 months ago
- ☆156Updated last year
- Layer-Condensed KV cache w/ 10 times larger batch size, fewer params and less computation. Dramatic speed up with better task performance…☆149Updated last month
- CUDA and Triton implementations of Flash Attention with SoftmaxN.☆70Updated last year
- A library for unit scaling in PyTorch☆125Updated 6 months ago
- ☆129Updated 3 months ago
- Cold Compress is a hackable, lightweight, and open-source toolkit for creating and benchmarking cache compression methods built on top of…☆132Updated 9 months ago
- Token Omission Via Attention☆126Updated 7 months ago
- ☆248Updated last year