proger / accelerated-scanLinks
Accelerated First Order Parallel Associative Scan
☆187Updated last year
Alternatives and similar repositories for accelerated-scan
Users that are interested in accelerated-scan are comparing it to the libraries listed below
Sorting:
- A library for unit scaling in PyTorch☆129Updated last month
- JAX bindings for Flash Attention v2☆90Updated 3 weeks ago
- Efficient optimizers☆254Updated 2 weeks ago
- ☆118Updated last year
- Code for exploring Based models from "Simple linear attention language models balance the recall-throughput tradeoff"☆240Updated 2 months ago
- A MAD laboratory to improve AI architecture designs 🧪☆124Updated 8 months ago
- ☆208Updated 8 months ago
- FlashFFTConv: Efficient Convolutions for Long Sequences with Tensor Cores☆327Updated 7 months ago
- Understand and test language model architectures on synthetic tasks.☆221Updated last month
- Experiment of using Tangent to autodiff triton☆80Updated last year
- ☆148Updated 2 years ago
- supporting pytorch FSDP for optimizers☆84Updated 8 months ago
- The simplest, fastest repository for training/finetuning medium-sized GPTs.☆152Updated last month
- ☆56Updated 10 months ago
- ☆65Updated 9 months ago
- ☆40Updated last year
- ☆53Updated last year
- seqax = sequence modeling + JAX☆166Updated last month
- 🧱 Modula software package☆222Updated 3 weeks ago
- CIFAR-10 speedruns: 94% in 2.6 seconds and 96% in 27 seconds☆283Updated last month
- Explorations into the recently proposed Taylor Series Linear Attention☆100Updated last year
- ☆81Updated last year
- ☆275Updated last year
- Normalized Transformer (nGPT)☆186Updated 9 months ago
- A fusion of a linear layer and a cross entropy loss, written for pytorch in triton.☆70Updated last year
- A PyTorch wrapper of parallel exclusive scan in CUDA☆12Updated 2 years ago
- A simple library for scaling up JAX programs☆143Updated 9 months ago
- 📄Small Batch Size Training for Language Models☆42Updated 2 weeks ago
- Simple implementation of muP, based on Spectral Condition for Feature Learning. The implementation is SGD only, dont use it for Adam☆85Updated last year
- Griffin MQA + Hawk Linear RNN Hybrid☆88Updated last year