ambisinister / mla-experimentsLinks
Experiments on Multi-Head Latent Attention
☆99Updated last year
Alternatives and similar repositories for mla-experiments
Users that are interested in mla-experiments are comparing it to the libraries listed below
Sorting:
- The simplest implementation of recent Sparse Attention patterns for efficient LLM inference.☆92Updated 6 months ago
- ☆83Updated 2 years ago
- Flash-Muon: An Efficient Implementation of Muon Optimizer☆229Updated 7 months ago
- ☆133Updated 8 months ago
- Code for studying the super weight in LLM☆120Updated last year
- The evaluation framework for training-free sparse attention in LLMs☆110Updated 3 months ago
- Fast and memory-efficient exact attention☆75Updated 10 months ago
- Low-bit optimizers for PyTorch☆137Updated 2 years ago
- Get down and dirty with FlashAttention2.0 in pytorch, plug in and play no complex CUDA kernels☆113Updated 2 years ago
- Transformers components but in Triton☆34Updated 8 months ago
- [ACL 2025] Squeezed Attention: Accelerating Long Prompt LLM Inference☆56Updated last year
- ☆104Updated 11 months ago
- ☆158Updated 11 months ago
- Simple and efficient pytorch-native transformer training and inference (batched)☆79Updated last year
- Unofficial implementation for the paper "Mixture-of-Depths: Dynamically allocating compute in transformer-based language models"☆176Updated last year
- NAACL '24 (Best Demo Paper RunnerUp) / MlSys @ NeurIPS '23 - RedCoast: A Lightweight Tool to Automate Distributed Training and Inference☆69Updated last year
- Awesome Triton Resources☆39Updated 9 months ago
- Implementation of Infini-Transformer in Pytorch☆112Updated last year
- ☆62Updated 2 years ago
- ☆150Updated 2 years ago
- Block Transformer: Global-to-Local Language Modeling for Fast Inference (NeurIPS 2024)☆163Updated 9 months ago
- GEAR: An Efficient KV Cache Compression Recipefor Near-Lossless Generative Inference of LLM☆176Updated last year
- [ICML 2025] From Low Rank Gradient Subspace Stabilization to Low-Rank Weights: Observations, Theories and Applications☆52Updated 3 months ago
- ☆54Updated last year
- MambaFormer in-context learning experiments and implementation for https://arxiv.org/abs/2402.04248☆58Updated last year
- The source code of our work "Prepacking: A Simple Method for Fast Prefilling and Increased Throughput in Large Language Models" [AISTATS …☆60Updated last year
- The official implementation of the paper "Towards Efficient Mixture of Experts: A Holistic Study of Compression Techniques (TMLR)".☆88Updated 10 months ago
- Flash Attention in 300-500 lines of CUDA/C++☆36Updated 5 months ago
- Cold Compress is a hackable, lightweight, and open-source toolkit for creating and benchmarking cache compression methods built on top of…☆148Updated last year
- Linear Attention Sequence Parallelism (LASP)☆88Updated last year