AlirezaMorsali / MLP-Attention
☆12Updated 8 months ago
Related projects: ⓘ
- ☆19Updated last month
- Semiring programming in PyTorch☆10Updated 7 months ago
- ☆22Updated 6 months ago
- Official repository of paper "RNNs Are Not Transformers (Yet): The Key Bottleneck on In-context Retrieval"☆24Updated 5 months ago
- ☆25Updated 5 months ago
- Reference implementation of "Softmax Attention with Constant Cost per Token" (Heinsen, 2024)☆22Updated 3 months ago
- ☆30Updated 8 months ago
- Official code for the paper "Attention as a Hypernetwork"☆20Updated 2 months ago
- Efficient PScan implementation in PyTorch☆15Updated 8 months ago
- HGRN2: Gated Linear RNNs with State Expansion☆46Updated last month
- ☆42Updated 7 months ago
- ☆28Updated this week
- Blog post☆16Updated 7 months ago
- ☆18Updated 3 months ago
- ☆24Updated 2 months ago
- Fine-Tuning Pre-trained Transformers into Decaying Fast Weights☆19Updated last year
- Parallel Associative Scan for Language Models☆16Updated 8 months ago
- Implementation of Recurrent Linear Transformers in Jax+Flax.☆14Updated 10 months ago
- Curse-of-memory phenomenon of RNNs in sequence modelling☆17Updated this week
- ☆16Updated last year
- Leveraging Recursive Gumbel-Max Trick for Approximate Inference in Combinatorial Spaces, NeurIPS 2021☆11Updated 2 years ago
- Code for the paper: https://arxiv.org/pdf/2309.06979.pdf☆14Updated last month
- Official code for "Accelerating Feedforward Computation via Parallel Nonlinear Equation Solving", ICML 2021☆25Updated 2 years ago
- [ICML 24 NGSM workshop] Associative Recurrent Memory Transformer implementation and scripts for training and evaluating☆26Updated last week
- [NeurIPS 2023 spotlight] Official implementation of HGRN in our NeurIPS 2023 paper - Hierarchically Gated Recurrent Neural Network for Se…☆60Updated 4 months ago
- ☆41Updated 2 months ago
- Official PyTorch Implementation of the Longhorn Deep State Space Model☆35Updated last month
- ☆14Updated 9 months ago
- [ICML 2023] "Data Efficient Neural Scaling Law via Model Reusing" by Peihao Wang, Rameswar Panda, Zhangyang Wang☆13Updated 8 months ago
- Parallelizing non-linear sequential models over the sequence length☆40Updated last month