Alternatives and similar repositories for kfac-pytorch

Users that are interested in kfac-pytorch are comparing it to the libraries listed below

• amirgholami / adahessian

  View on GitHub
  ADAHESSIAN: An Adaptive Second Order Optimizer for Machine Learning
  ☆283Feb 27, 2023Updated 2 years ago
• n-gao / pytorch-kfac

  View on GitHub
  Pytorch implementation of KFAC - this is a port of https://github.com/tensorflow/kfac/
  ☆30Jun 6, 2024Updated last year
• gpauloski / BERT-PyTorch

  View on GitHub
  BERT for Distributed PyTorch + AMP Training
  ☆12Mar 15, 2023Updated 2 years ago
• tyohei / chainerkfac

  View on GitHub
  A Chainer extension for K-FAC
  ☆20Jun 16, 2019Updated 6 years ago
• fL0n9 / SKFAC-MindSpore

  View on GitHub
  SKFAC Preconditioner for MindSpore
  ☆12Jul 2, 2021Updated 4 years ago
• Thrandis / EKFAC-pytorch

  View on GitHub
  Repository containing Pytorch code for EKFAC and K-FAC perconditioners.
  ☆152Jun 22, 2023Updated 2 years ago
• tfjgeorge / nngeometry

  View on GitHub
  {KFAC,EKFAC,Diagonal,Implicit} Fisher Matrices and finite width NTKs in PyTorch
  ☆221Updated this week
• cybertronai / pytorch-sso

  View on GitHub
  PyTorch-SSO: Scalable Second-Order methods in PyTorch
  ☆148Oct 1, 2023Updated 2 years ago
• tensorflow / kfac

  View on GitHub
  An implementation of KFAC for TensorFlow
  ☆199Feb 11, 2022Updated 4 years ago
• KordingLab / hilbert-constrained-gradient-descent

  View on GitHub
  Pytorch optimizers implementing Hilbert Constrained Gradient Descent
  ☆19May 9, 2019Updated 6 years ago
• gd-zhang / Weight-Decay

  View on GitHub
  Regularization, Neural Network Training Dynamics
  ☆14Jan 13, 2020Updated 6 years ago
• yaroslavvb / kfac_pytorch

  View on GitHub
  ☆135Oct 23, 2017Updated 8 years ago
• IST-DASLab / M-FAC

  View on GitHub
  Efficient reference implementations of the static & dynamic M-FAC algorithms (for pruning and optimization)
  ☆17Feb 23, 2022Updated 3 years ago
• rgrosse / csc2541_examples

  View on GitHub
  ☆30Feb 11, 2021Updated 5 years ago
• Kid-key / MimicNorm

  View on GitHub
  ☆19Jan 27, 2021Updated 5 years ago
• elizabethnewman / hessQuik

  View on GitHub
  Computing gradients and Hessians of feed-forward networks with GPU acceleration
  ☆20Feb 14, 2024Updated 2 years ago
• kunimi00 / ContrastiveSSLMusicAudio

  View on GitHub
  ☆13Jun 2, 2022Updated 3 years ago
• HKBU-HPML / OMGS-SGD

  View on GitHub
  Layer-wise Sparsification of Distributed Deep Learning
  ☆10Jul 6, 2020Updated 5 years ago
• SirRob1997 / Crowded-Valley---Results

  View on GitHub
  This repository contains the results for the paper: "Descending through a Crowded Valley - Benchmarking Deep Learning Optimizers"
  ☆184Jul 17, 2021Updated 4 years ago
• kazukiosawa / pipe-fisher

  View on GitHub
  ☆10Apr 29, 2023Updated 2 years ago
• apple / ml-tic-lm

  View on GitHub
  Repository for the paper: "TiC-LM: A Web-Scale Benchmark for Time-Continual LLM Pretraining" ACL Oral 2025
  ☆20Jan 31, 2026Updated 2 weeks ago
• LIJUNYI95 / SuperAdam

  View on GitHub
  Official Pytorch Implementation for the paper 'SUPER-ADAM: Faster and Universal Framework of Adaptive Gradients'
  ☆17Jan 12, 2022Updated 4 years ago
• uw-mad-dash / Accordion

  View on GitHub
  Code for reproducing experiments performed for Accoridon
  ☆13Jun 11, 2021Updated 4 years ago
• rwightman / py-lmdb

  View on GitHub
  Universal Python binding for the LMDB 'Lightning' Database
  ☆13Nov 7, 2017Updated 8 years ago
• Chavdarova / LAGAN-Lookahead_Minimax

  View on GitHub
  Source code for "Taming GANs with Lookahead–Minmax", ICLR 2021.
  ☆15Mar 28, 2021Updated 4 years ago
• Kaffaljidhmah2 / SpecDec_pp

  View on GitHub
  Repository for the COLM 2025 paper SpecDec++: Boosting Speculative Decoding via Adaptive Candidate Lengths
  ☆15Jul 10, 2025Updated 7 months ago
• locuslab / uniform-convergence-NeurIPS19

  View on GitHub
  The code for the NeurIPS19 paper and blog on "Uniform convergence may be unable to explain generalization in deep learning".
  ☆10Oct 26, 2019Updated 6 years ago
• Lifelong-ML / LASEM

  View on GitHub
  Code for the ICML 2021 paper "Sharing Less is More: Lifelong Learning in Deep Networks with Selective Layer Transfer"
  ☆12Aug 17, 2021Updated 4 years ago
• ParCIS / Ok-Topk

  View on GitHub
  Ok-Topk is a scheme for distributed training with sparse gradients. Ok-Topk integrates a novel sparse allreduce algorithm (less than 6k c…
  ☆27Dec 10, 2022Updated 3 years ago
• LGrCo / L-GreCo

  View on GitHub
  AN EFFICIENT AND GENERAL FRAMEWORK FOR LAYERWISE-ADAPTIVE GRADIENT COMPRESSION
  ☆14Oct 27, 2023Updated 2 years ago
• 2prime / OpenBlackBox

  View on GitHub
  ☆12Nov 5, 2019Updated 6 years ago
• lzhangbv / dear_pytorch

  View on GitHub
  [ICDCS 2023] DeAR: Accelerating Distributed Deep Learning with Fine-Grained All-Reduce Pipelining
  ☆12Dec 4, 2023Updated 2 years ago
• wiseodd / last_layer_laplace

  View on GitHub
  Last-layer Laplace approximation code examples
  ☆83Oct 18, 2021Updated 4 years ago
• githubharald / analyze_ada_hessian

  View on GitHub
  Analyze AdaHessian optimizer on 2D functions.
  ☆13Aug 13, 2021Updated 4 years ago
• linusericsson / ssl-invariances

  View on GitHub
  Official code for the paper "Why Do Self-Supervised Models Transfer? Investigating the Impact of Invariance on Downstream Tasks".
  ☆16Dec 7, 2021Updated 4 years ago
• rgrosse / fang

  View on GitHub
  code for experiments in Grosse and Salakhutdinov, 2015.
  ☆12Oct 9, 2016Updated 9 years ago
• mttmantovani / zotero-arxiv-doi-retriever

  View on GitHub
  Look for arXiv papers in a Zotero library and find available DOIs of published versions.
  ☆14Apr 11, 2024Updated last year
• Daniil-Selikhanovych / Shampoo_optimizer

  View on GitHub
  Our implementation of Shampoo optimizer based on https://arxiv.org/pdf/1802.09568.pdf
  ☆12Dec 23, 2019Updated 6 years ago
• Liuhong99 / Sophia

  View on GitHub
  The official implementation of “Sophia: A Scalable Stochastic Second-order Optimizer for Language Model Pre-training”
  ☆981Jan 30, 2024Updated 2 years ago