thomasahle / arithmetic-transformerLinks
Teaching Addition to Small Transformers
☆17Updated 2 years ago
Alternatives and similar repositories for arithmetic-transformer
Users that are interested in arithmetic-transformer are comparing it to the libraries listed below
Sorting:
- Fast Text Classification with Compressors dictionary☆150Updated 2 years ago
- Code implementing "Efficient Parallelization of a Ubiquitious Sequential Computation" (Heinsen, 2023)☆97Updated 11 months ago
- gzip Predicts Data-dependent Scaling Laws☆34Updated last year
- Official Repository of Pretraining Without Attention (BiGS), BiGS is the first model to achieve BERT-level transfer learning on the GLUE …☆115Updated last year
- Amos optimizer with JEstimator lib.☆82Updated last year
- Pre-train BERT from scratch, with HuggingFace. Accompanies the blog post: sidsite.com/posts/bert-from-scratch☆43Updated 6 months ago
- LayerNorm(SmallInit(Embedding)) in a Transformer to improve convergence☆61Updated 3 years ago
- slowly building a set of infinite riddle generators for data-hungry methods☆14Updated 3 years ago
- ☆72Updated last year
- AAAI 2022 Paper: Bet even Beth Harmon couldn't learn chess like that :)☆38Updated 4 years ago
- The GeoV model is a large langauge model designed by Georges Harik and uses Rotary Positional Embeddings with Relative distances (RoPER).…☆121Updated 2 years ago
- ☆105Updated 4 months ago
- Official repository for the paper "A Modern Self-Referential Weight Matrix That Learns to Modify Itself" (ICML 2022 & NeurIPS 2021 Deep R…☆172Updated 5 months ago
- Simplified implementation of UMAP like dimensionality reduction algorithm☆53Updated last year
- ☆29Updated 2 years ago
- Visualizing the internal board state of a GPT trained on chess PGN strings, and performing interventions on its internal board state and …☆219Updated last year
- DiCE: The Infinitely Differentiable Monte-Carlo Estimator☆32Updated 2 years ago
- This repository contain the simple llama3 implementation in pure jax.☆70Updated 9 months ago
- RWKV model implementation☆38Updated 2 years ago
- Various handy scripts to quickly setup new Linux and Windows sandboxes, containers and WSL.☆40Updated 2 weeks ago
- Evolution Pretraining Fully in Int Formats☆38Updated last week
- A minimal PyTorch re-implementation of GPT (Generative Pretrained Transformer) language model training☆17Updated 2 years ago
- Understanding how features learned by neural networks evolve throughout training☆39Updated last year
- JAX/Flax implementation of the Hyena Hierarchy☆34Updated 2 years ago
- Repo for solving arc problems with an Neural Cellular Automata☆22Updated 6 months ago
- ☆58Updated last week
- ☆31Updated 2 years ago
- a small code base for training large models☆315Updated 7 months ago
- Alice in Wonderland code base for experiments and raw experiments data☆131Updated 2 months ago
- ☆144Updated 2 years ago