Alternatives and similar repositories for pytorch-lamb

Users that are interested in pytorch-lamb are comparing it to the libraries listed below

• NUS-HPC-AI-Lab / LARS-ImageNet-PyTorch
  Accuracy 77%. Large batch deep learning optimizer LARS for ImageNet with PyTorch and ResNet, using Horovod for distribution. Optional acc…
  ☆38Updated 4 years ago
• HazyResearch / fly
  ☆219Updated 2 years ago
• zyushun / hessian-spectrum
  Code for the paper: Why Transformers Need Adam: A Hessian Perspective
  ☆65Updated 7 months ago
• libffcv / ffcv-imagenet
  Train ImageNet *fast* in 500 lines of code with FFCV
  ☆149Updated last year
• varunnair18 / FISH
  Code for "Training Neural Networks with Fixed Sparse Masks" (NeurIPS 2021).
  ☆59Updated 3 years ago
• epfml / dynamic-sparse-flash-attention
  ☆149Updated 2 years ago
• NUS-HPC-AI-Lab / oh-my-server
  ☆30Updated 2 years ago
• teelinsan / parallel-decoding
  Repository of the paper "Accelerating Transformer Inference for Translation via Parallel Decoding"
  ☆120Updated last year
• Lightning-AI / forked-pdb
  Python pdb for multiple processes
  ☆60Updated 5 months ago
• LiuXiaoxuanPKU / GACT-ICML
  ☆43Updated 3 years ago
• juntang-zhuang / GSAM
  PyTorch repository for ICLR 2022 paper (GSAM) which improves generalization (e.g. +3.8% top-1 accuracy on ImageNet with ViT-B/32)
  ☆144Updated 3 years ago
• pipilurj / BONAS
  ☆35Updated 4 years ago
• epfml / powersgd
  Practical low-rank gradient compression for distributed optimization: https://arxiv.org/abs/1905.13727
  ☆148Updated last year
• JeanKaddour / NoTrainNoGain
  Revisiting Efficient Training Algorithms For Transformer-based Language Models (NeurIPS 2023)
  ☆80Updated 2 years ago
• VITA-Group / SMC-Bench
  [ICLR 2023] "Sparsity May Cry: Let Us Fail (Current) Sparse Neural Networks Together!" Shiwei Liu, Tianlong Chen, Zhenyu Zhang, Xuxi Chen…
  ☆28Updated 2 years ago
• verbiiyo / rigl-torch
  Lightweight torch implementation of rigl, a sparse-to-sparse optimizer.
  ☆60Updated 3 years ago
• microsoft / Stochastic-Mixture-of-Experts
  This package implements THOR: Transformer with Stochastic Experts.
  ☆65Updated 4 years ago
• kashif / ICLR2022-OpenReviewData
  Crawl & visualize ICLR papers and reviews.
  ☆107Updated 3 years ago
• zhijian-liu / torchprofile
  A general and accurate MACs / FLOPs profiler for PyTorch models
  ☆629Updated 3 months ago
• facebookresearch / luckmatters
  Understanding Training Dynamics of Deep ReLU Networks
  ☆302Updated 3 weeks ago
• jianweif / OptimalGradCheckpointing
  ☆41Updated 4 years ago
• ronghanghu / vit_10b_fsdp_example
  See details in https://github.com/pytorch/xla/blob/r1.12/torch_xla/distributed/fsdp/README.md
  ☆25Updated 2 years ago
• JonasGeiping / fullbatchtraining
  Training vision models with full-batch gradient descent and regularization
  ☆39Updated 2 years ago
• IST-DASLab / WoodFisher
  Code accompanying the NeurIPS 2020 paper: WoodFisher (Singh & Alistarh, 2020)
  ☆53Updated 4 years ago
• yaozhewei / MLPruning
  MLPruning, PyTorch, NLP, BERT, Structured Pruning
  ☆20Updated 4 years ago
• locuslab / edge-of-stability
  ☆71Updated 11 months ago
• alexzhang13 / flashattention2-custom-mask
  Triton implementation of FlashAttention2 that adds Custom Masks.
  ☆146Updated last year
• WoosukKwon / retraining-free-pruning
  [NeurIPS 2022] A Fast Post-Training Pruning Framework for Transformers
  ☆192Updated 2 years ago
• weigq / iclr2022_stats
  ☆46Updated 2 years ago
• hwang595 / Cuttlefish
  The implementation for MLSys 2023 paper: "Cuttlefish: Low-rank Model Training without All The Tuning"
  ☆43Updated 2 years ago