alexzhang13 / flashattention2-custom-mask
Triton implementation of FlashAttention2 that adds Custom Masks.
☆91Updated 5 months ago
Alternatives and similar repositories for flashattention2-custom-mask:
Users that are interested in flashattention2-custom-mask are comparing it to the libraries listed below
- Triton-based implementation of Sparse Mixture of Experts.☆194Updated 2 months ago
- ☆108Updated 4 months ago
- A sparse attention kernel supporting mix sparse patterns☆98Updated 3 months ago
- ☆217Updated 8 months ago
- Odysseus: Playground of LLM Sequence Parallelism☆64Updated 7 months ago
- Official repository for LightSeq: Sequence Level Parallelism for Distributed Training of Long Context Transformers☆204Updated 5 months ago
- [ICML 2024] Quest: Query-Aware Sparsity for Efficient Long-Context LLM Inference☆236Updated 2 months ago
- Ring attention implementation with flash attention☆660Updated last month
- PyTorch bindings for CUTLASS grouped GEMM.☆61Updated 2 months ago
- 16-fold memory access reduction with nearly no loss☆72Updated 2 months ago
- An unofficial implementation of "Mixture-of-Depths: Dynamically allocating compute in transformer-based language models"☆35Updated 7 months ago
- Repository of the paper "Accelerating Transformer Inference for Translation via Parallel Decoding"☆114Updated 10 months ago
- Low-bit optimizers for PyTorch☆125Updated last year
- REST: Retrieval-Based Speculative Decoding, NAACL 2024☆190Updated last month
- This repository contains the experimental PyTorch native float8 training UX☆219Updated 5 months ago
- Code for Palu: Compressing KV-Cache with Low-Rank Projection☆64Updated this week
- GEAR: An Efficient KV Cache Compression Recipefor Near-Lossless Generative Inference of LLM☆152Updated 6 months ago
- A WebUI for Side-by-Side Comparison of Media (Images/Videos) Across Multiple Folders☆16Updated this week
- 🚀 Efficiently (pre)training foundation models with native PyTorch features, including FSDP for training and SDPA implementation of Flash…☆218Updated this week
- ☆55Updated 2 weeks ago
- PyTorch bindings for CUTLASS grouped GEMM.☆86Updated 3 weeks ago
- ☆97Updated 5 months ago
- The official implementation of the paper <MoA: Mixture of Sparse Attention for Automatic Large Language Model Compression>☆113Updated last month
- An algorithm for static activation quantization of LLMs☆112Updated 2 weeks ago
- Applied AI experiments and examples for PyTorch☆216Updated last week
- [MLSys'24] Atom: Low-bit Quantization for Efficient and Accurate LLM Serving☆291Updated 6 months ago
- [ICML 2024] KIVI: A Tuning-Free Asymmetric 2bit Quantization for KV Cache☆269Updated last week
- This repo contains the source code for: Model Tells You What to Discard: Adaptive KV Cache Compression for LLMs☆32Updated 5 months ago
- Fast and memory-efficient exact attention☆57Updated last month
- Fast Matrix Multiplications for Lookup Table-Quantized LLMs☆221Updated last week