JesseFarebro / flax-mupLinks
Maximal Update Parametrization (μP) with Flax & Optax.
☆16Updated last year
Alternatives and similar repositories for flax-mup
Users that are interested in flax-mup are comparing it to the libraries listed below
Sorting:
- Implementation of PSGD optimizer in JAX☆34Updated 7 months ago
- ☆115Updated last month
- LoRA for arbitrary JAX models and functions☆140Updated last year
- A simple library for scaling up JAX programs☆140Updated 9 months ago
- Pytorch-like dataloaders for JAX.☆92Updated 2 months ago
- Minimal but scalable implementation of large language models in JAX☆35Updated last week
- supporting pytorch FSDP for optimizers☆84Updated 7 months ago
- 🧱 Modula software package☆210Updated last week
- ☆141Updated this week
- Flow-matching algorithms in JAX☆100Updated 11 months ago
- Unofficial but Efficient Implementation of "Mamba: Linear-Time Sequence Modeling with Selective State Spaces" in JAX☆85Updated last year
- If it quacks like a tensor...☆58Updated 8 months ago
- ☆206Updated 8 months ago
- ☆31Updated 8 months ago
- Jax/Flax rewrite of Karpathy's nanoGPT☆59Updated 2 years ago
- Implementation of Gradient Agreement Filtering, from Chaubard et al. of Stanford, but for single machine microbatches, in Pytorch☆25Updated 6 months ago
- Efficient optimizers☆252Updated last week
- Einsum-like high-level array sharding API for JAX☆35Updated last year
- Pytorch implementation of preconditioned stochastic gradient descent (Kron and affine preconditioner, low-rank approximation precondition…☆180Updated last week
- 📄Small Batch Size Training for Language Models☆36Updated last week
- ☆274Updated last year
- ☆64Updated 8 months ago
- Implementation of Denoising Diffusion Probabilistic Models (DDPM) in JAX and Flax.☆21Updated last year
- JAX Synergistic Memory Inspector☆177Updated last year
- ☆53Updated 10 months ago
- Accelerated First Order Parallel Associative Scan☆184Updated 11 months ago
- A State-Space Model with Rational Transfer Function Representation.☆79Updated last year
- Scalable and Stable Parallelization of Nonlinear RNNS☆17Updated 6 months ago
- The simplest, fastest repository for training/finetuning medium-sized GPTs.☆35Updated last year
- nanoGPT using Equinox☆13Updated 2 years ago