Distributed K-FAC preconditioner for PyTorch
☆95Feb 25, 2026Updated last week
Alternatives and similar repositories for kfac-pytorch
Users that are interested in kfac-pytorch are comparing it to the libraries listed below
Sorting:
- Pytorch implementation of KFAC and E-KFAC (Natural Gradient).☆133Jul 2, 2019Updated 6 years ago
- ADAHESSIAN: An Adaptive Second Order Optimizer for Machine Learning☆283Feb 27, 2023Updated 3 years ago
- A Chainer extension for K-FAC☆20Jun 16, 2019Updated 6 years ago
- BackPACK - a backpropagation package built on top of PyTorch which efficiently computes quantities other than the gradient.☆606Nov 28, 2025Updated 3 months ago
- SKFAC Preconditioner for MindSpore☆12Jul 2, 2021Updated 4 years ago
- Repository containing Pytorch code for EKFAC and K-FAC perconditioners.☆153Jun 22, 2023Updated 2 years ago
- {KFAC,EKFAC,Diagonal,Implicit} Fisher Matrices and finite width NTKs in PyTorch☆221Feb 9, 2026Updated 3 weeks ago
- PyTorch-SSO: Scalable Second-Order methods in PyTorch☆148Oct 1, 2023Updated 2 years ago
- Second Order Optimization and Curvature Estimation with K-FAC in JAX.☆314Feb 25, 2026Updated last week
- PyHessian is a Pytorch library for second-order based analysis and training of Neural Networks☆777Jul 10, 2025Updated 7 months ago
- An implementation of KFAC for TensorFlow☆199Feb 11, 2022Updated 4 years ago
- Regularization, Neural Network Training Dynamics☆14Jan 13, 2020Updated 6 years ago
- Pytorch optimizers implementing Hilbert Constrained Gradient Descent☆19May 9, 2019Updated 6 years ago
- Efficient reference implementations of the static & dynamic M-FAC algorithms (for pruning and optimization)☆17Feb 23, 2022Updated 4 years ago
- ☆19Jan 27, 2021Updated 5 years ago
- Computing gradients and Hessians of feed-forward networks with GPU acceleration☆20Feb 14, 2024Updated 2 years ago
- Randomized algorithm class at CU☆15Jul 8, 2025Updated 7 months ago
- ☆13Jun 2, 2022Updated 3 years ago
- ☆33Jul 8, 2024Updated last year
- A PyTorch implement of Dilated RNN☆11Dec 31, 2017Updated 8 years ago
- Layer-wise Sparsification of Distributed Deep Learning☆10Jul 6, 2020Updated 5 years ago
- Repository for the paper: "TiC-LM: A Web-Scale Benchmark for Time-Continual LLM Pretraining" ACL Oral 2025☆21Feb 27, 2026Updated last week
- A decentralised application that creates high quality machine learning datasets☆13Jan 22, 2019Updated 7 years ago
- This repository contains the results for the paper: "Descending through a Crowded Valley - Benchmarking Deep Learning Optimizers"☆184Jul 17, 2021Updated 4 years ago
- ☆10Apr 29, 2023Updated 2 years ago
- Repository for the COLM 2025 paper SpecDec++: Boosting Speculative Decoding via Adaptive Candidate Lengths☆15Jul 10, 2025Updated 7 months ago
- awesome unsupervised learning paper list☆12Jan 4, 2018Updated 8 years ago
- [ICLR 2023] Eva: Practical Second-order Optimization with Kronecker-vectorized Approximation☆12Jul 31, 2023Updated 2 years ago
- The code for the NeurIPS19 paper and blog on "Uniform convergence may be unable to explain generalization in deep learning".☆10Oct 26, 2019Updated 6 years ago
- A computation-parallel deep learning architecture.☆13Sep 25, 2019Updated 6 years ago
- Source code for "Taming GANs with Lookahead–Minmax", ICLR 2021.☆15Mar 28, 2021Updated 4 years ago
- Code for the ICML 2021 paper "Sharing Less is More: Lifelong Learning in Deep Networks with Selective Layer Transfer"☆12Aug 17, 2021Updated 4 years ago
- Universal Python binding for the LMDB 'Lightning' Database☆13Nov 7, 2017Updated 8 years ago
- Artifact for IPDPS'21: DSXplore: Optimizing Convolutional Neural Networks via Sliding-Channel Convolutions.☆13Apr 6, 2021Updated 4 years ago
- Host CIFAR-10.2 Data Set☆13Sep 22, 2021Updated 4 years ago
- Ok-Topk is a scheme for distributed training with sparse gradients. Ok-Topk integrates a novel sparse allreduce algorithm (less than 6k c…☆27Dec 10, 2022Updated 3 years ago
- [CVPR 2021] Code release for "Unsupervised Feature Learning by Cross-Level Instance-Group Discrimination."☆101May 8, 2022Updated 3 years ago
- [ICDCS 2023] DeAR: Accelerating Distributed Deep Learning with Fine-Grained All-Reduce Pipelining☆12Dec 4, 2023Updated 2 years ago
- ☆12Nov 5, 2019Updated 6 years ago