hazan-lab / flash-stuLinks
PyTorch implementation of the Flash Spectral Transform Unit.
☆17Updated 9 months ago
Alternatives and similar repositories for flash-stu
Users that are interested in flash-stu are comparing it to the libraries listed below
Sorting:
- FlexAttention w/ FlashAttention3 Support☆26Updated 8 months ago
- Here we will test various linear attention designs.☆59Updated last year
- Code for the paper: https://arxiv.org/pdf/2309.06979.pdf☆19Updated 10 months ago
- Benchmark tests supporting the TiledCUDA library.☆16Updated 7 months ago
- Triton Implementation of HyperAttention Algorithm☆48Updated last year
- Code and data for paper "(How) do Language Models Track State?"☆14Updated 2 months ago
- Transformers components but in Triton☆34Updated last month
- ☆36Updated 2 months ago
- ☆32Updated 8 months ago
- Awesome Triton Resources☆31Updated last month
- Experiment of using Tangent to autodiff triton☆79Updated last year
- Parallel Associative Scan for Language Models☆18Updated last year
- Fast and memory-efficient exact attention☆68Updated 3 months ago
- ☆21Updated last month
- ☆19Updated last month
- The evaluation framework for training-free sparse attention in LLMs☆69Updated last week
- Triton kernels for Flux☆20Updated 5 months ago
- Efficient PScan implementation in PyTorch☆16Updated last year
- Make triton easier☆46Updated last year
- research impl of Native Sparse Attention (2502.11089)☆54Updated 4 months ago
- Train a SmolLM-style llm on fineweb-edu in JAX/Flax with an assortment of optimizers.☆17Updated 3 months ago
- Source-to-Source Debuggable Derivatives in Pure Python☆15Updated last year
- ☆22Updated last year
- ☆14Updated 11 months ago
- Implementation of Hyena Hierarchy in JAX☆10Updated 2 years ago
- ☆37Updated last year
- CUDA implementation of autoregressive linear attention, with all the latest research findings☆44Updated 2 years ago
- Using FlexAttention to compute attention with different masking patterns☆44Updated 9 months ago
- APPy (Annotated Parallelism for Python) enables users to annotate loops and tensor expressions in Python with compiler directives akin to…☆23Updated this week
- Xmixers: A collection of SOTA efficient token/channel mixers☆11Updated 7 months ago