Triton implementation of FlashAttention2 that adds Custom Masks.
☆180Aug 14, 2024Updated 2 years ago
Alternatives and similar repositories for flashattention2-custom-mask
Users that are interested in flashattention2-custom-mask are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Personal solutions to the Triton Puzzles☆22Jul 18, 2024Updated 2 years ago
- We develop world models that can be adapted with natural language. Intergrating these models into artificial agents allows humans to effe…☆25Feb 10, 2024Updated 2 years ago
- Helpful tools and examples for working with flex-attention☆1,264Updated this week
- ☆23Dec 1, 2021Updated 4 years ago
- Selected list of papers on World Models that I found interesting and/or useful.☆43Feb 8, 2025Updated last year
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- Ring attention implementation with flash attention☆1,067Sep 10, 2025Updated last year
- Efficient triton implementation of Native Sparse Attention.☆285May 23, 2025Updated last year
- Official repository for DistFlashAttn: Distributed Memory-efficient Attention for Long-context LLMs Training☆222Aug 19, 2024Updated 2 years ago
- Code for ICLR 2025 Paper "What is Wrong with Perplexity for Long-context Language Modeling?"☆118Oct 11, 2025Updated last year
- Fast and memory-efficient exact attention☆74Mar 3, 2025Updated last year
- Using FlexAttention to compute attention with different masking patterns☆47Sep 22, 2024Updated 2 years ago
- ☆17Jun 19, 2023Updated 3 years ago
- ☆13Apr 15, 2024Updated 2 years ago
- ☆152Jun 2, 2023Updated 3 years ago
- End-to-end encrypted email - Proton Mail • AdSpecial offer: 40% Off Yearly / 80% Off First Month. All Proton services are open source and independently audited for security.
- Code for NeurIPS 2022 Spotlight paper " Non-Monotonic Latent Alignments for CTC-Based Non-Autoregressive Machine Translation"☆20Nov 16, 2022Updated 3 years ago
- A sparse attention kernel supporting mix sparse patterns☆566Jan 18, 2026Updated 9 months ago
- [NeurIPS'24 Spotlight, ICLR'25, ICML'25] To speed up Long-context LLMs' inference, approximate and dynamic sparse calculate the attention…☆1,232Oct 10, 2026Updated 3 weeks ago
- Official repository for the paper "Blending Complementary Memory Systems in Hybrid Quadratic-Linear Transformers"☆13Jun 3, 2025Updated last year
- USP: Unified (a.k.a. Hybrid, 2D) Sequence Parallel Attention for Long Context Transformers Model Training and Inference☆696Sep 16, 2026Updated last month
- Official repository of paper "RNNs Are Not Transformers (Yet): The Key Bottleneck on In-context Retrieval"☆27Apr 17, 2024Updated 2 years ago
- ☆13Feb 7, 2023Updated 3 years ago
- Official Implementation for the ICLR2023 paper "Fuzzy Alignments in Directed Acyclic Graph for Non-autoregressive Machine Translation"☆14Mar 1, 2023Updated 3 years ago
- flash attention tutorial written in python, triton, cuda, cutlass☆542Oct 22, 2026Updated 2 weeks ago
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- QAQ: Quality Adaptive Quantization for LLM KV Cache☆55Mar 27, 2024Updated 2 years ago
- ☆20Dec 24, 2024Updated last year
- Cataloging released Triton kernels.☆315Sep 9, 2025Updated last year
- An agent for CUDA compute-communication kernel co-design☆37May 7, 2026Updated 6 months ago
- ☆107Sep 9, 2024Updated 2 years ago
- Implementation of FlashAttention (FA1-FA4) in PyTorch for educational and algorithmic clarity☆228Apr 12, 2026Updated 6 months ago
- Large Context Attention☆775Oct 13, 2025Updated last year
- ☆126May 28, 2024Updated 2 years ago
- Here we will test various linear attention designs.☆62Apr 25, 2024Updated 2 years ago
- AI Agents on DigitalOcean Gradient AI Platform • AdBuild production-ready AI agents using customizable tools or access multiple LLMs through a single endpoint. Create custom knowledge bases or connect external data.
- Benchmark tests supporting the TiledCUDA library.☆19Nov 19, 2024Updated last year
- The collections of MOE (Mixture Of Expert) papers, code and tools, etc.☆12Mar 15, 2024Updated 2 years ago
- [ICML 2025] XAttention: Block Sparse Attention with Antidiagonal Scoring☆288Jul 6, 2025Updated last year
- 高性能短序列稀疏Mask Attention CUDA算子，针对<1K序列+75%稀疏度优化☆79Mar 18, 2026Updated 7 months ago
- 🚀 Efficient implementations for emerging model architectures☆6,026Updated this week
- Sirius, an efficient correction mechanism, which significantly boosts Contextual Sparsity models on reasoning tasks while maintaining its…☆22Sep 10, 2024Updated 2 years ago
- A Suite for Parallel Inference of Diffusion Transformers (DiTs) on multi-GPU Clusters☆58May 3, 2026Updated 6 months ago