borjanG / 2023-transformersLinks
Codes for the paper The emergence of clusters in self-attention dynamics.
☆17Updated last year
Alternatives and similar repositories for 2023-transformers
Users that are interested in 2023-transformers are comparing it to the libraries listed below
Sorting:
- Omnigrok: Grokking Beyond Algorithmic Data☆62Updated 2 years ago
- ☆70Updated 10 months ago
- ☆58Updated last year
- ☆33Updated last year
- Parallelizing non-linear sequential models over the sequence length☆54Updated 3 months ago
- DoG is SGD's Best Friend: A Parameter-Free Dynamic Step Size Schedule☆63Updated 2 years ago
- Replicating and dissecting the git-re-basin project in one-click-replication Colabs☆35Updated 3 years ago
- Unofficial but Efficient Implementation of "Mamba: Linear-Time Sequence Modeling with Selective State Spaces" in JAX☆88Updated last year
- Blog post☆17Updated last year
- unofficial re-implementation of "Grokking: Generalization Beyond Overfitting on Small Algorithmic Datasets"☆79Updated 3 years ago
- Implementation of OpenAI's 'Grokking: Generalization Beyond Overfitting on Small Algorithmic Datasets' paper.☆39Updated 2 years ago
- A MAD laboratory to improve AI architecture designs 🧪☆129Updated 9 months ago
- ☆11Updated last year
- ☆40Updated last month
- ☆53Updated last year
- ☆34Updated last year
- Experiments on the impact of depth in transformers and SSMs.☆34Updated 11 months ago
- Code for the paper: "Tensor Programs II: Neural Tangent Kernel for Any Architecture"☆104Updated 5 years ago
- Transformers with doubly stochastic attention☆48Updated 3 years ago
- Efficient Riemannian Optimization on Stiefel Manifold via Cayley Transform☆42Updated 6 years ago
- ☆216Updated 10 months ago
- Sequence Modeling with Multiresolution Convolutional Memory (ICML 2023)☆127Updated last year
- Non official implementation of the Linear Recurrent Unit (LRU, Orvieto et al. 2023)☆57Updated last month
- Pytorch implementation of preconditioned stochastic gradient descent (Kron and affine preconditioner, low-rank approximation precondition…☆183Updated 2 weeks ago
- nanoGPT-like codebase for LLM training☆107Updated 4 months ago
- Sampling with gradient-based Markov Chain Monte Carlo approaches☆108Updated last year
- Universal Neurons in GPT2 Language Models☆30Updated last year
- Code accompanying our paper "Feature Learning in Infinite-Width Neural Networks" (https://arxiv.org/abs/2011.14522)☆63Updated 4 years ago
- Official implementation of Stochastic Taylor Derivative Estimator (STDE) NeurIPS2024☆122Updated 10 months ago
- Codes for the paper "A mathematical perspective on Transformers".☆38Updated last year