f-dangel / sirfshampoo
[ICML 2024] SIRFShampoo: Structured inverse- and root-free Shampoo in PyTorch (https://arxiv.org/abs/2402.03496)
☆14Updated 5 months ago
Alternatives and similar repositories for sirfshampoo:
Users that are interested in sirfshampoo are comparing it to the libraries listed below
- [ICML 2024] SINGD: KFAC-like Structured Inverse-Free Natural Gradient Descent (http://arxiv.org/abs/2312.05705)☆21Updated 5 months ago
- [TMLR 2022] Curvature access through the generalized Gauss-Newton's low-rank structure: Eigenvalues, eigenvectors, directional derivative…☆17Updated last year
- Layerwise Batch Entropy Regularization☆22Updated 2 years ago
- Official code for Long Expressive Memory (ICLR 2022, Spotlight)☆69Updated 3 years ago
- The accompanying code for "Simplifying and Understanding State Space Models with Diagonal Linear RNNs" (Ankit Gupta, Harsh Mehta, Jonatha…☆20Updated 2 years ago
- ☆36Updated last year
- PyTorch linear operators for curvature matrices (Hessian, Fisher/GGN, KFAC, ...)☆36Updated last month
- Transformers with doubly stochastic attention☆45Updated 2 years ago
- code for "Semi-Discrete Normalizing Flows through Differentiable Tessellation"☆26Updated 2 years ago
- Code for the article "What if Neural Networks had SVDs?", to be presented as a spotlight paper at NeurIPS 2020.☆75Updated 8 months ago
- Jupyter Notebook corresponding to 'Going with the Flow: An Introduction to Normalizing Flows'☆25Updated 3 years ago
- ☆20Updated 2 years ago
- Limitations of the Empirical Fisher Approximation☆47Updated last month
- AdaCat☆49Updated 2 years ago
- Sequence Modeling with Multiresolution Convolutional Memory (ICML 2023)☆123Updated last year
- ☆22Updated 5 months ago
- Pytorch implementation of preconditioned stochastic gradient descent (Kron and affine preconditioner, low-rank approximation precondition…☆171Updated last week
- ☆33Updated 2 years ago
- Code for ICLR 2021 Paper, "Anytime Sampling for Autoregressive Models via Ordered Autoencoding"☆26Updated last year
- ☆16Updated last year
- PyTorch utilities for ML, specifically speech☆13Updated last year
- Sequence Modeling with Structured State Spaces☆63Updated 2 years ago
- ☆52Updated 6 months ago
- Easy-to-use AdaHessian optimizer (PyTorch)☆78Updated 4 years ago
- A home for audio ML in JAX. Has common features, learnable frontends, pretrained supervised and self-supervised models.☆67Updated 2 years ago
- ☆49Updated 4 years ago
- A PyTorch wrapper of parallel exclusive scan in CUDA☆12Updated last year
- Supplementary code for the paper "Meta-Solver for Neural Ordinary Differential Equations" https://arxiv.org/abs/2103.08561☆24Updated 4 years ago
- ☆64Updated 7 months ago
- Convolutions and more as einsum for PyTorch☆16Updated 10 months ago