nikhilvyas / SOAPLinks
β188Updated 6 months ago
Alternatives and similar repositories for SOAP
Users that are interested in SOAP are comparing it to the libraries listed below
Sorting:
- Efficient optimizersβ212Updated last week
- π§± Modula software packageβ200Updated 2 months ago
- Pytorch implementation of preconditioned stochastic gradient descent (Kron and affine preconditioner, low-rank approximation preconditionβ¦β176Updated last week
- supporting pytorch FSDP for optimizersβ82Updated 6 months ago
- Accelerated First Order Parallel Associative Scanβ182Updated 10 months ago
- The AdEMAMix Optimizer: Better, Faster, Older.β183Updated 9 months ago
- An implementation of PSGD Kron second-order optimizer for PyTorchβ91Updated 2 months ago
- The simplest, fastest repository for training/finetuning medium-sized GPTs.β134Updated this week
- CIFAR-10 speedruns: 94% in 2.6 seconds and 96% in 27 secondsβ251Updated 3 months ago
- A MAD laboratory to improve AI architecture designs π§ͺβ120Updated 6 months ago
- β61Updated 7 months ago
- β270Updated 11 months ago
- LoRA for arbitrary JAX models and functionsβ138Updated last year
- Just some miscellaneous utility functions / decorators / modules related to Pytorch and Accelerate to help speed up implementation of newβ¦β122Updated 10 months ago
- β99Updated last week
- β150Updated 10 months ago
- β53Updated 8 months ago
- Flow-matching algorithms in JAXβ96Updated 10 months ago
- Implementation of PSGD optimizer in JAXβ33Updated 5 months ago
- Pytorch-like dataloaders for JAX.β86Updated 3 weeks ago
- Implementation of GateLoop Transformer in Pytorch and Jaxβ89Updated last year
- β27Updated last year
- Simple implementation of muP, based on Spectral Condition for Feature Learning. The implementation is SGD only, dont use it for Adamβ80Updated 10 months ago
- β78Updated 11 months ago
- For optimization algorithm research and development.β521Updated this week
- Implementation of Diffusion Transformer (DiT) in JAXβ278Updated last year
- Diffusion models in PyTorchβ104Updated this week
- A library for unit scaling in PyTorchβ125Updated 6 months ago
- A State-Space Model with Rational Transfer Function Representation.β78Updated last year
- Implementation of the proposed Adam-atan2 from Google Deepmind in Pytorchβ108Updated 6 months ago