borjanG / 2023-transformersLinks
Codes for the paper The emergence of clusters in self-attention dynamics.
☆17Updated last year
Alternatives and similar repositories for 2023-transformers
Users that are interested in 2023-transformers are comparing it to the libraries listed below
Sorting:
- ☆56Updated 10 months ago
- Omnigrok: Grokking Beyond Algorithmic Data☆61Updated 2 years ago
- ☆32Updated 10 months ago
- Unofficial but Efficient Implementation of "Mamba: Linear-Time Sequence Modeling with Selective State Spaces" in JAX☆87Updated last year
- Pytorch implementation of preconditioned stochastic gradient descent (Kron and affine preconditioner, low-rank approximation precondition…☆179Updated last week
- ☆207Updated 8 months ago
- Sequence Modeling with Multiresolution Convolutional Memory (ICML 2023)☆125Updated last year
- Parallelizing non-linear sequential models over the sequence length☆53Updated 2 months ago
- Non official implementation of the Linear Recurrent Unit (LRU, Orvieto et al. 2023)☆56Updated last month
- Implementation of OpenAI's 'Grokking: Generalization Beyond Overfitting on Small Algorithmic Datasets' paper.☆39Updated last year
- Replicating and dissecting the git-re-basin project in one-click-replication Colabs☆36Updated 2 years ago
- Transformers with doubly stochastic attention☆47Updated 2 years ago
- Simple (and cheap!) neural network uncertainty estimation☆69Updated 2 months ago
- Code for experiments on transformers using Markovian data.☆19Updated 9 months ago
- A State-Space Model with Rational Transfer Function Representation.☆79Updated last year
- Implementations of various linear RNN layers using pytorch and triton☆53Updated 2 years ago
- The Energy Transformer block, in JAX☆59Updated last year
- Blog post☆17Updated last year
- DoG is SGD's Best Friend: A Parameter-Free Dynamic Step Size Schedule☆63Updated 2 years ago
- Codes for the paper "A mathematical perspective on Transformers".☆38Updated last year
- A MAD laboratory to improve AI architecture designs 🧪☆125Updated 8 months ago
- 📄Small Batch Size Training for Language Models☆42Updated 2 weeks ago
- ☆70Updated 8 months ago
- Code for the paper: "Tensor Programs II: Neural Tangent Kernel for Any Architecture"☆105Updated 4 years ago
- Why Do We Need Weight Decay in Modern Deep Learning? [NeurIPS 2024]☆67Updated 11 months ago
- About A collection of AWESOME things about information geometry Topics☆165Updated last year
- Pytorch code for experiments on Linear Transformers☆21Updated last year
- ☆182Updated last year
- ☆37Updated last year
- ☆34Updated last year