xie-lab-ml / deep-learning-dynamics-paper-list
This is a list of peer-reviewed representative papers on deep learning dynamics (optimization dynamics of neural networks). The success of deep learning attributes to both network architecture and stochastic optimization. Thus, deep learning dynamics play an essentially important role in theoretical foundation of deep learning.
☆273Updated last year
Alternatives and similar repositories for deep-learning-dynamics-paper-list:
Users that are interested in deep-learning-dynamics-paper-list are comparing it to the libraries listed below
- [ICML 2022, Oral] The PyTorch Implementation of Adaptive Inertia Methods. The algorithms are based on our paper: "Adaptive Inertia: Dise…☆150Updated 2 years ago
- Welcome to the Awesome Feature Learning in Deep Learning Thoery Reading Group! This repository serves as a collaborative platform for sch…☆179Updated 4 months ago
- Neural Tangent Kernel Papers☆110Updated 3 months ago
- A curated list of papers of interesting empirical study and insight on deep learning. Continually updating...☆323Updated 3 weeks ago
- ☆47Updated 7 months ago
- Welcome to the 'In Context Learning Theory' Reading Group☆27Updated 6 months ago
- summer school materials☆44Updated last year
- Collecting research materials on EBM/EBL (Energy Based Models, Energy Based Learning)☆293Updated last year
- Open-L2O: A Comprehensive and Reproducible Benchmark for Learning to Optimize Algorithms☆277Updated last year
- a collection of AWESOME things about Optimal Transport in Deep Learning☆267Updated last year
- Code for the paper: Why Transformers Need Adam: A Hessian Perspective☆57Updated 2 months ago
- This is a curated list for Information Bottleneck Principle, in memory of Professor Naftali Tishby.☆354Updated 11 months ago
- ☆224Updated 2 years ago
- ☆256Updated 2 months ago
- [NeurIPS 2023] The PyTorch Implementation of Scheduled (Stable) Weight Decay.☆60Updated last year
- [ICML 2021] The official PyTorch Implementations of Positive-Negative Momentum Optimizers.☆28Updated 2 years ago
- PyTorch implementation for Vision Transformer[Dosovitskiy, A.(ICLR'21)] modified to obtain over 90% accuracy FROM SCRATCH on CIFAR-10 wit…☆191Updated last year
- PyHessian is a Pytorch library for second-order based analysis and training of Neural Networks☆735Updated last year
- A lecture note for understanding deep learning☆317Updated last month
- [ICML 2024] Official code for the paper "Revisiting Zeroth-Order Optimization for Memory-Efficient LLM Fine-Tuning: A Benchmark ".☆102Updated 10 months ago
- Bi-level Optimization for Advanced Deep Learning☆45Updated 3 years ago
- Official Implementation of Learning Diverse and Discriminative Representations via the Principle of Maximal Coding Rate Reduction (2020)☆198Updated 2 years ago
- TorchOpt is an efficient library for differentiable optimization built upon PyTorch.☆584Updated this week
- [Neural Computation, MIT Press] The PyTorch Implementation of Variable Optimizers/ Neural Variable Risk Minimization proposed in our Neur…☆33Updated 3 years ago
- Example code for paper "Bilevel Optimization: Nonasymptotic Analysis and Faster Algorithms"☆47Updated 3 years ago
- Approximating neural network loss landscapes in low-dimensional parameter subspaces for PyTorch☆329Updated last year
- Official implementation for Estimating the Optimal Covariance with Imperfect Mean in Diffusion Probabilistic Models (ICML 2022), and a re…☆109Updated 2 years ago
- Create animations for the optimization trajectory of neural nets☆153Updated last year
- ☆66Updated 6 years ago
- Collect optimizer related papers, data, repositories☆90Updated 5 months ago