thomasahle / arithmetic-transformerLinks
Teaching Addition to Small Transformers
☆17Updated 2 years ago
Alternatives and similar repositories for arithmetic-transformer
Users that are interested in arithmetic-transformer are comparing it to the libraries listed below
Sorting:
- Code implementing "Efficient Parallelization of a Ubiquitious Sequential Computation" (Heinsen, 2023)☆98Updated last year
- gzip Predicts Data-dependent Scaling Laws☆34Updated last year
- Fast Text Classification with Compressors dictionary☆150Updated 2 years ago
- LayerNorm(SmallInit(Embedding)) in a Transformer to improve convergence☆61Updated 3 years ago
- Learning Universal Predictors☆81Updated last year
- DiCE: The Infinitely Differentiable Monte-Carlo Estimator☆32Updated 2 years ago
- Various handy scripts to quickly setup new Linux and Windows sandboxes, containers and WSL.☆40Updated last week
- RWKV model implementation☆38Updated 2 years ago
- Jax like function transformation engine but micro, microjax☆34Updated last year
- ☆255Updated 2 years ago
- The GeoV model is a large langauge model designed by Georges Harik and uses Rotary Positional Embeddings with Relative distances (RoPER).…☆121Updated 2 years ago
- Amos optimizer with JEstimator lib.☆82Updated last year
- JAX/Flax implementation of the Hyena Hierarchy☆34Updated 2 years ago
- Official Repository of Pretraining Without Attention (BiGS), BiGS is the first model to achieve BERT-level transfer learning on the GLUE …☆116Updated last year
- A place to store reusable transformer components of my own creation or found on the interwebs☆70Updated 2 weeks ago
- Yet another random morning idea to be quickly tried and architecture shared if it works; to allow the transformer to pause for any amount…☆53Updated 2 years ago
- ☆144Updated 2 years ago
- Pre-train BERT from scratch, with HuggingFace. Accompanies the blog post: sidsite.com/posts/bert-from-scratch☆43Updated 8 months ago
- Demonstration that finetuning RoPE model on larger sequences than the pre-trained model adapts the model context limit☆63Updated 2 years ago
- Evolution Pretraining Fully in Int Formats☆135Updated last month
- Alice in Wonderland code base for experiments and raw experiments data☆131Updated 4 months ago
- ☆29Updated 2 years ago
- ☆59Updated 2 months ago
- A minimal PyTorch re-implementation of GPT (Generative Pretrained Transformer) language model training☆18Updated 2 years ago
- ☆109Updated 6 months ago
- Latent Diffusion Language Models☆70Updated 2 years ago
- Repo for solving arc problems with an Neural Cellular Automata☆23Updated 8 months ago
- ☆35Updated 2 years ago
- Training code for Sparse Autoencoders on Embedding models☆39Updated 11 months ago
- Simplified implementation of UMAP like dimensionality reduction algorithm☆53Updated last year