google-deepmind / recurrentgemmaLinks
Open weights language model from Google DeepMind, based on Griffin.
☆645Updated last month
Alternatives and similar repositories for recurrentgemma
Users that are interested in recurrentgemma are comparing it to the libraries listed below
Sorting:
- a small code base for training large models☆307Updated 3 months ago
- [ICLR 2025] Samba: Simple Hybrid State Space Models for Efficient Unlimited Context Language Modeling☆899Updated 3 months ago
- Annotated version of the Mamba paper☆487Updated last year
- Fast bare-bones BPE for modern tokenizer training☆160Updated last month
- Minimalistic, extremely fast, and hackable researcher's toolbench for GPT models in 307 lines of code. Reaches <3.8 validation loss on wi…☆349Updated last year
- Repo for "Monarch Mixer: A Simple Sub-Quadratic GEMM-Based Architecture"☆555Updated 7 months ago
- Visualize the intermediate output of Mistral 7B☆367Updated 6 months ago
- Official codebase for the paper "Beyond A* Better Planning with Transformers via Search Dynamics Bootstrapping".☆372Updated last year
- The repository for the code of the UltraFastBERT paper☆516Updated last year
- Code to train and evaluate Neural Attention Memory Models to obtain universally-applicable memory systems for transformers.☆316Updated 9 months ago
- ☆191Updated this week
- Accelerate, Optimize performance with streamlined training and serving options with JAX.☆293Updated this week
- Legible, Scalable, Reproducible Foundation Models with Named Tensors and Jax☆627Updated this week
- A complete end-to-end pipeline for LLM interpretability with sparse autoencoders (SAEs) using Llama 3.2, written in pure PyTorch and full…☆620Updated 4 months ago
- ☆274Updated last year
- Implementation of 💍 Ring Attention, from Liu et al. at Berkeley AI, in Pytorch☆532Updated 2 months ago
- ☆304Updated last year
- Reference implementation of Megalodon 7B model☆522Updated 2 months ago
- Official repository for the paper "Grokfast: Accelerated Grokking by Amplifying Slow Gradients"☆557Updated last year
- Inference code for Persimmon-8B☆415Updated last year
- ☆516Updated last year
- Memory layers use a trainable key-value lookup mechanism to add extra parameters to a model without increasing FLOPs. Conceptually, spars…☆342Updated 7 months ago
- For optimization algorithm research and development.☆525Updated this week
- Understand and test language model architectures on synthetic tasks.☆221Updated 2 weeks ago
- A subset of PyTorch's neural network modules, written in Python using OpenAI's Triton.☆563Updated 2 weeks ago
- Pax is a Jax-based machine learning framework for training large scale models. Pax allows for advanced and fully configurable experimenta…☆517Updated last week
- A Jax-based library for building transformers, includes implementations of GPT, Gemma, LlaMa, Mixtral, Whisper, SWin, ViT and more.☆291Updated 11 months ago
- A pure NumPy implementation of Mamba.☆223Updated last year
- Visualizing the internal board state of a GPT trained on chess PGN strings, and performing interventions on its internal board state and …☆208Updated 8 months ago
- [ICML 2024] CLLMs: Consistency Large Language Models☆397Updated 8 months ago