GindaChen / FlexFlashAttention3Links
FlexAttention w/ FlashAttention3 Support
☆27Updated 10 months ago
Alternatives and similar repositories for FlexFlashAttention3
Users that are interested in FlexFlashAttention3 are comparing it to the libraries listed below
Sorting:
- APPy (Annotated Parallelism for Python) enables users to annotate loops and tensor expressions in Python with compiler directives akin to…☆24Updated 2 months ago
- Implementation of Hyena Hierarchy in JAX☆10Updated 2 years ago
- Experiment of using Tangent to autodiff triton☆80Updated last year
- Make triton easier☆47Updated last year
- CUDA and Triton implementations of Flash Attention with SoftmaxN.☆73Updated last year
- Benchmark tests supporting the TiledCUDA library.☆17Updated 9 months ago
- ☆77Updated 3 months ago
- ☆32Updated last year
- Code and data for paper "(How) do Language Models Track State?"☆17Updated 4 months ago
- Awesome Triton Resources☆33Updated 3 months ago
- PyTorch implementation of the Flash Spectral Transform Unit.☆17Updated 11 months ago
- The evaluation framework for training-free sparse attention in LLMs☆90Updated 2 months ago
- Using FlexAttention to compute attention with different masking patterns☆44Updated 11 months ago
- [NeurIPS 2023] Sparse Modular Activation for Efficient Sequence Modeling☆39Updated last year
- Personal solutions to the Triton Puzzles☆20Updated last year
- Here we will test various linear attention designs.☆62Updated last year
- [WIP] Better (FP8) attention for Hopper☆33Updated 6 months ago
- A bunch of kernels that might make stuff slower 😉☆58Updated this week
- Fast and memory-efficient exact attention☆70Updated 5 months ago
- ☆22Updated 3 months ago
- Linear Attention Sequence Parallelism (LASP)☆86Updated last year
- ☆21Updated 5 months ago
- Flash-Muon: An Efficient Implementation of Muon Optimizer☆160Updated 2 months ago
- Transformers components but in Triton☆34Updated 3 months ago
- Parallel Associative Scan for Language Models☆18Updated last year
- An efficient implementation of the NSA (Native Sparse Attention) kernel☆113Updated 2 months ago
- Framework to reduce autotune overhead to zero for well known deployments.☆81Updated last week
- extensible collectives library in triton☆88Updated 4 months ago
- Reference implementation of "Softmax Attention with Constant Cost per Token" (Heinsen, 2024)☆24Updated last year
- train with kittens!☆62Updated 10 months ago