Alternatives and similar repositories for pytorch-lamb

Users that are interested in pytorch-lamb are comparing it to the libraries listed below

• NUS-HPC-AI-Lab / LARS-ImageNet-PyTorch
  Accuracy 77%. Large batch deep learning optimizer LARS for ImageNet with PyTorch and ResNet, using Horovod for distribution. Optional acc…
  ☆38Updated 4 years ago
• HazyResearch / fly
  ☆221Updated 2 years ago
• zyushun / hessian-spectrum
  Code for the paper: Why Transformers Need Adam: A Hessian Perspective
  ☆63Updated 9 months ago
• LiuXiaoxuanPKU / GACT-ICML
  ☆43Updated 3 years ago
• NUS-HPC-AI-Lab / oh-my-server
  ☆30Updated 2 years ago
• VITA-Group / SMC-Bench
  [ICLR 2023] "Sparsity May Cry: Let Us Fail (Current) Sparse Neural Networks Together!" Shiwei Liu, Tianlong Chen, Zhenyu Zhang, Xuxi Chen…
  ☆28Updated 2 years ago
• epfml / powersgd
  Practical low-rank gradient compression for distributed optimization: https://arxiv.org/abs/1905.13727
  ☆149Updated last year
• alexzhang13 / flashattention2-custom-mask
  Triton implementation of FlashAttention2 that adds Custom Masks.
  ☆155Updated last year
• libffcv / ffcv-imagenet
  Train ImageNet *fast* in 500 lines of code with FFCV
  ☆149Updated last year
• epfml / dynamic-sparse-flash-attention
  ☆150Updated 2 years ago
• gpauloski / kfac-pytorch
  Distributed K-FAC preconditioner for PyTorch
  ☆93Updated this week
• varunnair18 / FISH
  Code for "Training Neural Networks with Fixed Sparse Masks" (NeurIPS 2021).
  ☆59Updated 3 years ago
• juntang-zhuang / GSAM
  PyTorch repository for ICLR 2022 paper (GSAM) which improves generalization (e.g. +3.8% top-1 accuracy on ImageNet with ViT-B/32)
  ☆144Updated 3 years ago
• yxli2123 / LoSparse
  ☆62Updated 2 years ago
• teelinsan / parallel-decoding
  Repository of the paper "Accelerating Transformer Inference for Translation via Parallel Decoding"
  ☆121Updated last year
• Lightning-AI / forked-pdb
  Python pdb for multiple processes
  ☆70Updated 6 months ago
• pipilurj / BONAS
  ☆35Updated 4 years ago
• JonasGeiping / linear_cross_entropy_loss
  A fusion of a linear layer and a cross entropy loss, written for pytorch in triton.
  ☆73Updated last year
• VITA-Group / Junk_DNA_Hypothesis
  [ICML 2024] Junk DNA Hypothesis: A Task-Centric Angle of LLM Pre-trained Weights through Sparsity; Lu Yin*, Ajay Jaiswal*, Shiwei Liu, So…
  ☆16Updated 8 months ago
• hushon / JAX-ResNet-CIFAR10
  Simple CIFAR10 ResNet example with JAX.
  ☆23Updated 4 years ago
• zbh2047 / L_inf-dist-net
  [ICML 2021] This is the official github repo for training L_inf dist nets with high certified accuracy.
  ☆42Updated 3 years ago
• verbiiyo / rigl-torch
  Lightweight torch implementation of rigl, a sparse-to-sparse optimizer.
  ☆60Updated 4 years ago
• nblt / DLDR
  [TPAMI 2023] Low Dimensional Landscape Hypothesis is True: DNNs can be Trained in Tiny Subspaces
  ☆42Updated 3 years ago
• jianweif / OptimalGradCheckpointing
  ☆41Updated 4 years ago
• JeanKaddour / NoTrainNoGain
  Revisiting Efficient Training Algorithms For Transformer-based Language Models (NeurIPS 2023)
  ☆81Updated 2 years ago
• microsoft / Stochastic-Mixture-of-Experts
  This package implements THOR: Transformer with Stochastic Experts.
  ☆65Updated 4 years ago
• ZO-Bench / ZO-LLM
  [ICML‘24] Official code for the paper "Revisiting Zeroth-Order Optimization for Memory-Efficient LLM Fine-Tuning: A Benchmark ".
  ☆120Updated 5 months ago
• Shiweiliuiiiiiii / In-Time-Over-Parameterization
  [ICML 2021] "Do We Actually Need Dense Over-Parameterization? In-Time Over-Parameterization in Sparse Training" by Shiwei Liu, Lu Yin, De…
  ☆45Updated 2 years ago
• Jiacheng-Zhu-AIML / AsymmetryLoRA
  Preprint: Asymmetry in Low-Rank Adapters of Foundation Models
  ☆37Updated last year
• RulinShao / LightSeq
  Official repository for DistFlashAttn: Distributed Memory-efficient Attention for Long-context LLMs Training
  ☆219Updated last year