xiayuqing0622 / flex_head_fa
Fast and memory-efficient exact attention
☆58Updated this week
Alternatives and similar repositories for flex_head_fa:
Users that are interested in flex_head_fa are comparing it to the libraries listed below
- ☆77Updated last year
- ☆138Updated last year
- Triton-based implementation of Sparse Mixture of Experts.☆196Updated 2 months ago
- Stick-breaking attention☆43Updated last month
- The source code of our work "Prepacking: A Simple Method for Fast Prefilling and Increased Throughput in Large Language Models"☆59Updated 4 months ago
- The simplest implementation of recent Sparse Attention patterns for efficient LLM inference.☆57Updated 3 weeks ago
- Linear Attention Sequence Parallelism (LASP)☆77Updated 8 months ago
- Boosting 4-bit inference kernels with 2:4 Sparsity☆64Updated 5 months ago
- Here we will test various linear attention designs.☆58Updated 9 months ago
- Triton Implementation of HyperAttention Algorithm☆46Updated last year
- Transformers components but in Triton☆31Updated 3 months ago
- ☆111Updated this week
- ☆51Updated 9 months ago
- Tree Attention: Topology-aware Decoding for Long-Context Attention on GPU clusters☆116Updated 2 months ago
- ☆44Updated 3 months ago
- Pruner-Zero: Evolving Symbolic Pruning Metric from scratch for LLMs☆77Updated 2 months ago
- A MAD laboratory to improve AI architecture designs 🧪☆102Updated 2 months ago
- Understand and test language model architectures on synthetic tasks.☆181Updated last month
- Code for exploring Based models from "Simple linear attention language models balance the recall-throughput tradeoff"☆221Updated this week
- GEAR: An Efficient KV Cache Compression Recipefor Near-Lossless Generative Inference of LLM☆157Updated 7 months ago
- [ICLR 2024 Spotlight] Code for the paper "Merge, Then Compress: Demystify Efficient SMoE with Hints from Its Routing Policy"☆72Updated 8 months ago
- ☆82Updated 4 months ago
- Simple and efficient pytorch-native transformer training and inference (batched)☆68Updated 10 months ago
- ☆91Updated 8 months ago
- Token Omission Via Attention☆123Updated 4 months ago
- Repository for Sparse Finetuning of LLMs via modified version of the MosaicML llmfoundry☆40Updated last year
- Code for NeurIPS 2024 Spotlight: "Scaling Laws and Compute-Optimal Training Beyond Fixed Training Durations"☆70Updated 3 months ago
- Official implementation of the ICML 2024 paper RoSA (Robust Adaptation)☆38Updated last year
- Implementation of Infini-Transformer in Pytorch☆109Updated last month
- Experiments on Multi-Head Latent Attention☆67Updated 6 months ago