zoq / Awesome-Optimizer
Collect optimizer related papers, data, repositories
☆76Updated 9 months ago
Related projects: ⓘ
- Distributed K-FAC Preconditioner for PyTorch☆75Updated last week
- Neural Tangent Kernel Papers☆84Updated 6 months ago
- TensorLy-Torch: Deep Tensor Learning with TensorLy and PyTorch☆72Updated 3 months ago
- summer school materials☆46Updated last year
- A thoroughly investigated survey for tensorial neural networks.☆103Updated 11 months ago
- ☆12Updated 3 years ago
- Implementation of "Gradients without backpropagation" paper (https://arxiv.org/abs/2202.08587) using functorch☆94Updated last year
- ☆56Updated 3 years ago
- Create animations for the optimization trajectory of neural nets☆133Updated 7 months ago
- Repo for the paper "Landscape Surrogate Learning Decision Losses for Mathematical Optimization Under Partial Information"☆33Updated last year
- {KFAC,EKFAC,Diagonal,Implicit} Fisher Matrices and finite width NTKs in PyTorch☆204Updated last month
- Apply CP, Tucker, TT/TR, HT to compress neural networks. Train from scratch.☆13Updated 3 years ago
- [ICLR'24] "DeepZero: Scaling up Zeroth-Order Optimization for Deep Model Training" by Aochuan Chen*, Yimeng Zhang*, Jinghan Jia, James Di…☆35Updated 6 months ago
- Lightweight torch implementation of rigl, a sparse-to-sparse optimizer.☆55Updated 2 years ago
- Non official implementation of the Linear Recurrent Unit (LRU, Orvieto et al. 2023)☆47Updated 7 months ago
- ☆32Updated last year
- Efficient Riemannian Optimization on Stiefel Manifold via Cayley Transform☆34Updated 5 years ago
- ☆28Updated last year
- ☆65Updated 5 years ago
- Pytorch implementation of KFAC and E-KFAC (Natural Gradient).☆125Updated 5 years ago
- Sampling with gradient-based Markov Chain Monte Carlo approaches☆86Updated 5 months ago
- Benchmarking optimization methods on convex problems.☆27Updated 6 months ago
- SLTrain: a sparse plus low-rank approach for parameter and memory efficient pretraining☆15Updated 2 months ago
- Deep Learning & Information Bottleneck☆45Updated last year
- Optimal Transport in the Big Data Era☆86Updated 5 months ago
- Second-Order Neural ODE Optimizer, NeurIPS 2021 spotlight☆52Updated 2 years ago
- PyTorch implementation of Hessian Free optimisation☆43Updated 4 years ago
- Hessian backpropagation (HBP): PyTorch extension of backpropagation for block-diagonal curvature matrix approximations☆20Updated last year
- ☆187Updated last year
- Code to simulate energy-based neuromorphic platforms and learning algorithms for such platforms☆10Updated 3 months ago