ethansmith2000 / fsdp_optimizersLinks
supporting pytorch FSDP for optimizers
☆84Updated last year
Alternatives and similar repositories for fsdp_optimizers
Users that are interested in fsdp_optimizers are comparing it to the libraries listed below
Sorting:
- ☆91Updated last year
- WIP☆93Updated last year
- Efficient optimizers☆277Updated this week
- Simple implementation of muP, based on Spectral Condition for Feature Learning. The implementation is SGD only, dont use it for Adam☆84Updated last year
- Minimal (400 LOC) implementation Maximum (multi-node, FSDP) GPT training☆132Updated last year
- The simplest, fastest repository for training/finetuning medium-sized GPTs.☆180Updated 5 months ago
- ☆53Updated last year
- Fast, Modern, and Low Precision PyTorch Optimizers☆118Updated 3 months ago
- ☆30Updated last year
- Supporting code for the blog post on modular manifolds.☆107Updated 2 months ago
- Focused on fast experimentation and simplicity☆76Updated 11 months ago
- A library for unit scaling in PyTorch☆133Updated 5 months ago
- DeMo: Decoupled Momentum Optimization☆197Updated last year
- MoE training for Me and You and maybe other people☆239Updated this week
- research impl of Native Sparse Attention (2502.11089)☆63Updated 10 months ago
- Accelerated First Order Parallel Associative Scan☆193Updated last year
- ☆229Updated last year
- ☆68Updated last year
- Experiment of using Tangent to autodiff triton☆81Updated last year
- 🧱 Modula software package☆316Updated 4 months ago
- ☆34Updated last year
- A MAD laboratory to improve AI architecture designs 🧪☆135Updated last year
- ☆121Updated last year
- seqax = sequence modeling + JAX☆169Updated 5 months ago
- Understand and test language model architectures on synthetic tasks.☆246Updated 2 months ago
- ☆62Updated last year
- Tiny re-implementation of MDM in style of LLaDA and nano-gpt speedrun☆56Updated 9 months ago
- H-Net Dynamic Hierarchical Architecture☆80Updated 3 months ago
- ☆82Updated last year
- ☆107Updated last week