cloneofsimo / minSAELinks
☆28Updated 6 months ago
Alternatives and similar repositories for minSAE
Users that are interested in minSAE are comparing it to the libraries listed below
Sorting:
- Simple implementation of muP, based on Spectral Condition for Feature Learning. The implementation is SGD only, dont use it for Adam☆80Updated 10 months ago
- supporting pytorch FSDP for optimizers☆82Updated 6 months ago
- WIP☆93Updated 10 months ago
- Minimal (400 LOC) implementation Maximum (multi-node, FSDP) GPT training☆127Updated last year
- Focused on fast experimentation and simplicity☆74Updated 6 months ago
- Tiny re-implementation of MDM in style of LLaDA and nano-gpt speedrun☆52Updated 3 months ago
- ☆78Updated 11 months ago
- ☆19Updated last month
- research impl of Native Sparse Attention (2502.11089)☆54Updated 4 months ago
- Minimal but scalable implementation of large language models in JAX☆35Updated 7 months ago
- ☆36Updated 2 months ago
- Maximal Update Parametrization (μP) with Flax & Optax.☆11Updated last year
- The simplest, fastest repository for training/finetuning medium-sized GPTs.☆134Updated this week
- ☆24Updated last month
- ☆47Updated 4 months ago
- Minimal (truly) muP implementation, consistent with TP4 and TP5 papers notation☆14Updated last month
- ☆34Updated 9 months ago
- ☆51Updated last year
- ☆21Updated 7 months ago
- Mixture of A Million Experts☆46Updated 10 months ago
- ☆53Updated last year
- Exploration into the proposed "Self Reasoning Tokens" by Felipe Bonetto☆56Updated last year
- ☆20Updated 8 months ago
- ☆23Updated last year
- ☆61Updated 7 months ago
- ☆52Updated last year
- ☆79Updated 10 months ago
- ☆104Updated last week
- Fast, Modern, and Low Precision PyTorch Optimizers☆94Updated this week
- ☆21Updated 2 weeks ago