Alternatives and similar repositories for pytorch-lamb:

Users that are interested in pytorch-lamb are comparing it to the libraries listed below

• NUS-HPC-AI-Lab / LARS-ImageNet-PyTorch
  Accuracy 77%. Large batch deep learning optimizer LARS for ImageNet with PyTorch and ResNet, using Horovod for distribution. Optional acc…
  ☆38Updated 3 years ago
• Jiacheng-Zhu-AIML / AsymmetryLoRA
  Preprint: Asymmetry in Low-Rank Adapters of Foundation Models
  ☆31Updated 11 months ago
• dguo98 / DiffPruning
  Parameter Efficient Transfer Learning with Diff Pruning
  ☆73Updated 3 years ago
• LeapLabTHU / Deep-Incubation
  Code release for Deep Incubation (https://arxiv.org/abs/2212.04129)
  ☆91Updated last year
• yxli2123 / LoSparse
  ☆49Updated last year
• zyushun / hessian-spectrum
  Code for the paper: Why Transformers Need Adam: A Hessian Perspective
  ☆48Updated 9 months ago
• Lucky-Lance / SPP
  [ICML 2024] SPP: Sparsity-Preserved Parameter-Efficient Fine-Tuning for Large Language Models
  ☆18Updated 8 months ago
• ZO-Bench / ZO-LLM
  [ICML 2024] Official code for the paper "Revisiting Zeroth-Order Optimization for Memory-Efficient LLM Fine-Tuning: A Benchmark ".
  ☆83Updated 7 months ago
• teelinsan / parallel-decoding
  Repository of the paper "Accelerating Transformer Inference for Translation via Parallel Decoding"
  ☆114Updated 10 months ago
• HazyResearch / fly
  ☆200Updated last year
• varunnair18 / FISH
  Code for "Training Neural Networks with Fixed Sparse Masks" (NeurIPS 2021).
  ☆58Updated 3 years ago
• yaozhewei / MLPruning
  MLPruning, PyTorch, NLP, BERT, Structured Pruning
  ☆21Updated 3 years ago
• VITA-Group / SMC-Bench
  [ICLR 2023] "Sparsity May Cry: Let Us Fail (Current) Sparse Neural Networks Together!" Shiwei Liu, Tianlong Chen, Zhenyu Zhang, Xuxi Chen…
  ☆27Updated last year
• hwang595 / Cuttlefish
  The implementation for MLSys 2023 paper: "Cuttlefish: Low-rank Model Training without All The Tuning"
  ☆43Updated last year
• microsoft / Stochastic-Mixture-of-Experts
  This package implements THOR: Transformer with Stochastic Experts.
  ☆61Updated 3 years ago
• huyz2023 / 2by4-pretrain
  Efficient 2:4 sparse training algorithms and implementations
  ☆46Updated last month
• DS3Lab / AC-SGD
  Code associated with the paper **Fine-tuning Language Models over Slow Networks using Activation Compression with Guarantees**.
  ☆27Updated last year
• LiuXiaoxuanPKU / GACT-ICML
  ☆41Updated 2 years ago
• streethagore / ForwardLocalGradient
  This is the official implementation of the ICML 2023 paper - Can Forward Gradient Match Backpropagation ?
  ☆12Updated last year
• juntang-zhuang / GSAM
  PyTorch repository for ICLR 2022 paper (GSAM) which improves generalization (e.g. +3.8% top-1 accuracy on ImageNet with ViT-B/32)
  ☆139Updated 2 years ago
• JonasGeiping / fullbatchtraining
  Training vision models with full-batch gradient descent and regularization
  ☆37Updated last year
• papers-submission / structured_transposable_masks
  Code for ICML 2021 submission
  ☆35Updated 3 years ago
• pipilurj / BONAS
  ☆35Updated 3 years ago
• huggingface / block_movement_pruning
  Block Sparse movement pruning
  ☆78Updated 4 years ago
• JeanKaddour / NoTrainNoGain
  Revisiting Efficient Training Algorithms For Transformer-based Language Models (NeurIPS 2023)
  ☆79Updated last year
• JonasGeiping / linear_cross_entropy_loss
  A fusion of a linear layer and a cross entropy loss, written for pytorch in triton.
  ☆61Updated 5 months ago
• jianweif / OptimalGradCheckpointing
  ☆40Updated 3 years ago
• NUS-HPC-AI-Lab / oh-my-server
  ☆30Updated last year
• zyxxmu / DSnoT
  Official Pytorch Implementation of Our Paper Accepted at ICLR 2024-- Dynamic Sparse No Training: Training-Free Fine-tuning for Sparse LLM…
  ☆38Updated 9 months ago
• tml-epfl / sam-low-rank-features
  Sharpness-Aware Minimization Leads to Low-Rank Features [NeurIPS 2023]
  ☆25Updated last year