NX-AI / flashrnnLinks
FlashRNN - Fast RNN Kernels with I/O Awareness
☆98Updated 3 months ago
Alternatives and similar repositories for flashrnn
Users that are interested in flashrnn are comparing it to the libraries listed below
Sorting:
- Tiled Flash Linear Attention library for fast and efficient mLSTM Kernels.☆70Updated last month
- ☆251Updated 4 months ago
- Flash-Muon: An Efficient Implementation of Muon Optimizer☆189Updated 3 months ago
- Normalized Transformer (nGPT)☆190Updated 10 months ago
- Accelerated First Order Parallel Associative Scan☆190Updated last year
- [ICLR 2025] Official PyTorch Implementation of Gated Delta Networks: Improving Mamba2 with Delta Rule☆312Updated 3 weeks ago
- Fast and memory-efficient exact attention☆70Updated 7 months ago
- Tree Attention: Topology-aware Decoding for Long-Context Attention on GPU clusters☆130Updated 10 months ago
- The evaluation framework for training-free sparse attention in LLMs☆100Updated 3 months ago
- Pytorch implementation of the PEER block from the paper, Mixture of A Million Experts, by Xu Owen He at Deepmind☆129Updated last year
- Load compute kernels from the Hub☆293Updated this week
- Work in progress.☆74Updated 3 months ago
- This repository contains the experimental PyTorch native float8 training UX☆224Updated last year
- ☆67Updated 10 months ago
- ring-attention experiments☆152Updated 11 months ago
- Experiment of using Tangent to autodiff triton☆81Updated last year
- JAX bindings for Flash Attention v2☆95Updated 3 weeks ago
- Quick implementation of nGPT, learning entirely on the hypersphere, from NvidiaAI☆292Updated 4 months ago
- RWKV-7: Surpassing GPT☆96Updated 10 months ago
- A bunch of kernels that might make stuff slower 😉☆59Updated this week
- An implementation of PSGD Kron second-order optimizer for PyTorch☆95Updated 2 months ago
- CUDA and Triton implementations of Flash Attention with SoftmaxN.☆73Updated last year
- ☆34Updated last year
- PyTorch implementation of models from the Zamba2 series.☆185Updated 8 months ago
- Official implementation of the paper: "ZClip: Adaptive Spike Mitigation for LLM Pre-Training".☆133Updated last month
- 📄Small Batch Size Training for Language Models☆62Updated last week
- ☆57Updated last year
- ☆129Updated 4 months ago
- A MAD laboratory to improve AI architecture designs 🧪☆129Updated 9 months ago
- Griffin MQA + Hawk Linear RNN Hybrid☆89Updated last year