Alternatives and similar repositories for moshpit-sgd

Users that are interested in moshpit-sgd are comparing it to the libraries listed below

• juntang-zhuang / ACProp-Optimizer
  Implementation for ACProp ( Momentum centering and asynchronous update for adaptive gradient methdos, NeurIPS 2021)
  ☆15Updated 3 years ago
• HazyResearch / mongoose
  A Learnable LSH Framework for Efficient NN Training
  ☆31Updated 3 years ago
• petuum / tuun
  Hyperparameter tuning via uncertainty modeling
  ☆47Updated last year
• DS3Lab / AC-SGD
  Code associated with the paper **Fine-tuning Language Models over Slow Networks using Activation Compression with Guarantees**.
  ☆28Updated 2 years ago
• mryab / learning-at-home
  "Towards Crowdsourced Training of Large Neural Networks using Decentralized Mixture-of-Experts" (NeurIPS 2020), original PyTorch implemen…
  ☆56Updated 4 years ago
• nick11roberts / XD
  ☆12Updated 2 years ago
• lucidrains / kronecker-attention-pytorch
  Implementation of Kronecker Attention in Pytorch
  ☆19Updated 4 years ago
• minhtannguyen / SRSGD
  Code base for SRSGD.
  ☆28Updated 5 years ago
• learning-at-home / lean_transformer
  Memory-efficient transformer. Work in progress.
  ☆19Updated 2 years ago
• Distributed-AI / PipeTransformer
  PipeTransformer: Automated Elastic Pipelining for Distributed Training of Large-scale Models. ICML 2021
  ☆56Updated 3 years ago
• liamcli / darts_asha
  Code release to reproduce ASHA experiments from "Random Search and Reproducibility for NAS."
  ☆22Updated 5 years ago
• shwinshaker / LipGrow
  An adaptive training algorithm for residual network
  ☆15Updated 4 years ago
• sjunhongshen / DASH
  ☆23Updated 2 years ago
• Ying1123 / llm-caching-multiplexing
  ☆20Updated 2 years ago
• jiaweizzhao / signSGD-with-Majority-Vote
  ☆74Updated 6 years ago
• wolfecameron / optimizers
  implements various optimizers from scratch for analysis and comparison
  ☆9Updated 5 years ago
• DS3Lab / CocktailSGD
  ☆27Updated last year
• microsoft / fnl_paper
  Factorized Neural Layers
  ☆29Updated last year
• JeanKaddour / LAWA
  Latest Weight Averaging (NeurIPS HITY 2022)
  ☆30Updated 2 years ago
• google-research / wide-sparse-nets
  ☆19Updated 4 years ago
• JingzhaoZhang / why-clipping-accelerates
  A pytorch implementation for the LSTM experiments in the paper: Why Gradient Clipping Accelerates Training: A Theoretical Justification f…
  ☆46Updated 5 years ago
• YuhanLiu11 / AutoFreeze
  ☆22Updated 4 years ago
• facebookresearch / spartan
  Spartan is an algorithm for training sparse neural network models. This repository accompanies the paper "Spartan Differentiable Sparsity…
  ☆24Updated 2 years ago
• ChristophReich1996 / HyperMixer
  PyTorch reimplementation of the paper "HyperMixer: An MLP-based Green AI Alternative to Transformers" [arXiv 2022].
  ☆17Updated 3 years ago
• lucidrains / autoregressive-linear-attention-cuda
  CUDA implementation of autoregressive linear attention, with all the latest research findings
  ☆44Updated 2 years ago
• yandex-research / btard
  Code for the paper "Secure Distributed Training at Scale" (ICML 2022)
  ☆16Updated 4 months ago
• fjxmlzn / GPUTaskScheduler
  GPU Task Scheduler (Python library)
  ☆43Updated 4 years ago
• tanyuqian / redco
  NAACL '24 (Best Demo Paper RunnerUp) / MlSys @ NeurIPS '23 - RedCoast: A Lightweight Tool to Automate Distributed Training and Inference
  ☆66Updated 6 months ago
• JYWa / Overlap_Local_SGD
  Implementation of (overlap) local SGD in Pytorch
  ☆33Updated 4 years ago
• JeanKaddour / NoTrainNoGain
  Revisiting Efficient Training Algorithms For Transformer-based Language Models (NeurIPS 2023)
  ☆80Updated last year