f-dangel / sirfshampoo
[ICML 2024] SIRFShampoo: Structured inverse- and root-free Shampoo in PyTorch (https://arxiv.org/abs/2402.03496)
☆14Updated 2 months ago
Alternatives and similar repositories for sirfshampoo:
Users that are interested in sirfshampoo are comparing it to the libraries listed below
- [TMLR 2022] Curvature access through the generalized Gauss-Newton's low-rank structure: Eigenvalues, eigenvectors, directional derivative…☆17Updated last year
- [ICML 2024] SINGD: KFAC-like Structured Inverse-Free Natural Gradient Descent (http://arxiv.org/abs/2312.05705)☆21Updated 2 months ago
- Layerwise Batch Entropy Regularization☆22Updated 2 years ago
- The accompanying code for "Simplifying and Understanding State Space Models with Diagonal Linear RNNs" (Ankit Gupta, Harsh Mehta, Jonatha…☆19Updated 2 years ago
- Pytorch implementation of preconditioned stochastic gradient descent (Kron and affine preconditioner, low-rank approximation precondition…☆146Updated last month
- An implementation of PSGD Kron second-order optimizer for PyTorch☆21Updated 2 weeks ago
- A home for audio ML in JAX. Has common features, learnable frontends, pretrained supervised and self-supervised models.☆64Updated 2 years ago
- PyTorch linear operators for curvature matrices (Hessian, Fisher/GGN, KFAC, ...)☆24Updated this week
- ☆36Updated 11 months ago
- Official code for Long Expressive Memory (ICLR 2022, Spotlight)☆69Updated 2 years ago
- ☆63Updated 4 months ago
- Transformers with doubly stochastic attention☆44Updated 2 years ago
- ☆33Updated last year
- Test pytorch code with minimal computational overhead☆25Updated last year
- Sequence Modeling with Multiresolution Convolutional Memory (ICML 2023)☆121Updated last year
- A minimal Pytorch Implementation of Stochastically Quantized Variational AutoEncoder (SQ-VAE) by Sony☆30Updated last year
- code for "Semi-Discrete Normalizing Flows through Differentiable Tessellation"☆24Updated 2 years ago
- ☆22Updated 2 months ago
- ☆15Updated last year
- Jax/Flax implementation of Variational-DiffWave.☆40Updated 2 years ago
- Jupyter Notebook corresponding to 'Going with the Flow: An Introduction to Normalizing Flows'☆25Updated 3 years ago
- Code repository of the paper "Wavelet Networks: Scale-Translation Equivariant Learning From Raw Time-Series, TMLR" https://arxiv.org/abs…☆82Updated 11 months ago
- PyTorch utilities for ML, specifically speech☆13Updated 11 months ago
- ☆36Updated 2 years ago
- Continuous Augmented Positional Embeddings (CAPE) implementation for PyTorch☆39Updated 2 years ago
- ☆9Updated last year
- Efficient Householder Transformation in PyTorch☆63Updated 3 years ago
- A repo based on XiLin Li's PSGD repo that extends some of the experiments.☆14Updated 3 months ago
- Why Do We Need Weight Decay in Modern Deep Learning? [NeurIPS 2024]☆58Updated 3 months ago
- A PyTorch wrapper of parallel exclusive scan in CUDA☆9Updated last year