A port of muP to JAX/Haiku
☆25Oct 23, 2022Updated 3 years ago
Alternatives and similar repositories for haiku-mup
Users that are interested in haiku-mup are comparing it to the libraries listed below
Sorting:
- ☆18Aug 24, 2024Updated last year
- Code for the NLP4Prog workshop paper "Reading StackOverflow Encourages Cheating: Adding Question TextImproves Extractive Code Generation"☆21Aug 10, 2021Updated 4 years ago
- High-performance tokenized language data-loader for Python C++ extension☆14Jul 22, 2024Updated last year
- JAX Scalify: end-to-end scaled arithmetics☆18Oct 30, 2024Updated last year
- ☆13Dec 15, 2025Updated 2 months ago
- [Oral; Neurips OPT2024 ] μLO: Compute-Efficient Meta-Generalization of Learned Optimizers☆15Feb 12, 2026Updated 3 weeks ago
- nanoGPT using Equinox☆15Mar 3, 2023Updated 3 years ago
- LoRA for arbitrary JAX models and functions☆145Feb 26, 2024Updated 2 years ago
- Schedule free optimiser implemented in JAX using Optimistix☆15May 29, 2024Updated last year
- Unofficial JAX implementation of the SOAP optimizer (https://arxiv.org/abs/2409.11321)☆24Jan 9, 2026Updated 2 months ago
- Dreamer on JAX☆16Jan 19, 2022Updated 4 years ago
- The accompanying code for "Exploring the limits of decoder-only models trained on public speech recognition corpora" (Ankit Gupta, George…☆20Oct 11, 2024Updated last year
- Code for the paper "Function-Space Learning Rates"☆25Jun 3, 2025Updated 9 months ago
- My NixOS Configuration.☆20Mar 2, 2026Updated last week
- A simple implementation of Hamiltonian Monte Carlo in JAX.☆20Feb 8, 2024Updated 2 years ago
- GPT2 Byte Pair Encoding implementation in Golang☆24Jul 9, 2025Updated 8 months ago
- Unofficial but Efficient Implementation of "Mamba: Linear-Time Sequence Modeling with Selective State Spaces" in JAX☆93Jan 25, 2024Updated 2 years ago
- ☆23Jun 18, 2024Updated last year
- [ACL'24 Oral] Analysing The Impact of Sequence Composition on Language Model Pre-Training☆23Aug 18, 2024Updated last year
- Transformer with Mu-Parameterization, implemented in Jax/Flax. Supports FSDP on TPU pods.☆32Jun 5, 2025Updated 9 months ago
- Experiments on the impact of depth in transformers and SSMs.☆41Oct 23, 2025Updated 4 months ago
- ☆32Jan 7, 2024Updated 2 years ago
- An annotated implementation of the Hyena Hierarchy paper☆34May 28, 2023Updated 2 years ago
- [ICLR 2025] SDTT: a simple and effective distillation method for discrete diffusion models☆47Feb 26, 2026Updated last week
- Engineering the state of RNN language models (Mamba, RWKV, etc.)☆32May 25, 2024Updated last year
- ☆29Feb 27, 2024Updated 2 years ago
- ☆28Nov 18, 2022Updated 3 years ago
- ☆33Jan 14, 2021Updated 5 years ago
- ☆35Apr 12, 2024Updated last year
- Exponential families for JAX☆76Mar 3, 2026Updated last week
- Voice swapping with VQ-VAE and diffusion models☆68Nov 8, 2021Updated 4 years ago
- A python algorithm to change the pitch of the voice in real time☆13Dec 13, 2020Updated 5 years ago
- [ICML 2024] Official Repository for the paper "Transformers Get Stable: An End-to-End Signal Propagation Theory for Language Models"☆10Jul 19, 2024Updated last year
- Einsum-like high-level array sharding API for JAX☆34Jul 16, 2024Updated last year
- RWKV-X is a Linear Complexity Hybrid Language Model based on the RWKV architecture, integrating Sparse Attention to improve the model's l…☆55Updated this week
- Linear Attention Sequence Parallelism (LASP)☆89Jun 4, 2024Updated last year
- ☆44Jun 19, 2024Updated last year
- ☆16Jul 23, 2023Updated 2 years ago
- ACCO: An optimization algorithm for sharded distributed LLM training.☆13May 22, 2025Updated 9 months ago