danielmamay / grokkingLinks
Implementation of OpenAI's 'Grokking: Generalization Beyond Overfitting on Small Algorithmic Datasets' paper.
☆37Updated last year
Alternatives and similar repositories for grokking
Users that are interested in grokking are comparing it to the libraries listed below
Sorting:
- unofficial re-implementation of "Grokking: Generalization Beyond Overfitting on Small Algorithmic Datasets"☆78Updated 2 years ago
- Omnigrok: Grokking Beyond Algorithmic Data☆58Updated 2 years ago
- ☆29Updated 3 months ago
- Unofficial but Efficient Implementation of "Mamba: Linear-Time Sequence Modeling with Selective State Spaces" in JAX☆83Updated last year
- ☆26Updated 2 years ago
- Experiments on the impact of depth in transformers and SSMs.☆31Updated 7 months ago
- Evaluation of neuro-symbolic engines☆35Updated 10 months ago
- ☆45Updated last year
- ☆68Updated 6 months ago
- A centralized place for deep thinking code and experiments☆84Updated last year
- Language models scale reliably with over-training and on downstream tasks☆97Updated last year
- ☆53Updated last year
- ☆83Updated last year
- Sparse Autoencoder Training Library☆52Updated last month
- Universal Neurons in GPT2 Language Models☆29Updated last year
- Official repository for our paper, Transformers Learn Higher-Order Optimization Methods for In-Context Learning: A Study with Linear Mode…☆16Updated 7 months ago
- Code to reproduce "Transformers Can Do Arithmetic with the Right Embeddings", McLeish et al (NeurIPS 2024)☆190Updated last year
- [ICLR 2025] Code for the paper "Beyond Autoregression: Discrete Diffusion for Complex Reasoning and Planning"☆64Updated 4 months ago
- ☆37Updated last year
- Code for NeurIPS 2024 Spotlight: "Scaling Laws and Compute-Optimal Training Beyond Fixed Training Durations"☆74Updated 7 months ago
- ☆180Updated last year
- Investigating the generalization behavior of LM probes trained to predict truth labels: (1) from one annotator to another, and (2) from e…☆27Updated last year
- Repository for NPHardEval, a quantified-dynamic benchmark of LLMs☆54Updated last year
- Code Release for "Broken Neural Scaling Laws" (BNSL) paper☆59Updated last year
- ☆28Updated last year
- Code associated to papers on superposition (in ML interpretability)☆28Updated 2 years ago
- ☆34Updated last year
- ☆32Updated 5 months ago
- nanoGPT-like codebase for LLM training☆98Updated last month
- A MAD laboratory to improve AI architecture designs 🧪☆122Updated 6 months ago