hazan-lab / flash-stuLinks
PyTorch implementation of the Flash Spectral Transform Unit.
☆18Updated last year
Alternatives and similar repositories for flash-stu
Users that are interested in flash-stu are comparing it to the libraries listed below
Sorting:
- FlexAttention w/ FlashAttention3 Support☆27Updated 11 months ago
- APPy (Annotated Parallelism for Python) enables users to annotate loops and tensor expressions in Python with compiler directives akin to…☆24Updated 3 months ago
- Awesome Triton Resources☆34Updated 5 months ago
- ☆22Updated 4 months ago
- Transformers components but in Triton☆34Updated 4 months ago
- ☆22Updated last year
- Reference implementation of "Softmax Attention with Constant Cost per Token" (Heinsen, 2024)☆24Updated last year
- Flash-Linear-Attention models beyond language☆17Updated last month
- Here we will test various linear attention designs.☆62Updated last year
- Efficient PScan implementation in PyTorch☆16Updated last year
- Benchmark tests supporting the TiledCUDA library.☆17Updated 10 months ago
- Parallel Associative Scan for Language Models☆18Updated last year
- Fast and memory-efficient exact attention☆70Updated 6 months ago
- ☆32Updated last year
- ☆41Updated 5 months ago
- Code and data for paper "(How) do Language Models Track State?"☆18Updated 6 months ago
- A bunch of kernels that might make stuff slower 😉☆59Updated this week
- AdaSplash: Adaptive Sparse Flash Attention (aka Flash Entmax Attention)☆26Updated 2 months ago
- The evaluation framework for training-free sparse attention in LLMs☆98Updated 3 months ago
- An efficient implementation of the NSA (Native Sparse Attention) kernel☆115Updated 3 months ago
- MambaFormer in-context learning experiments and implementation for https://arxiv.org/abs/2402.04248☆57Updated last year
- Framework to reduce autotune overhead to zero for well known deployments.☆84Updated last week
- Quantized Attention on GPU☆44Updated 10 months ago
- FlashRNN - Fast RNN Kernels with I/O Awareness☆98Updated 3 months ago
- Expanding linear RNN state-transition matrix eigenvalues to include negatives improves state-tracking tasks and language modeling without…☆17Updated 6 months ago
- Experiment of using Tangent to autodiff triton☆81Updated last year
- Triton implement of bi-directional (non-causal) linear attention☆56Updated 7 months ago
- Triton Implementation of HyperAttention Algorithm☆48Updated last year
- continous batching and parallel acceleration for RWKV6☆23Updated last year
- Flash-Muon: An Efficient Implementation of Muon Optimizer☆189Updated 3 months ago