NUS-HPC-AI-Lab / pytorch-lamb

Related projects ⓘ

Alternatives and complementary repositories for pytorch-lamb

• NUS-HPC-AI-Lab / LARS-ImageNet-PyTorch
  Accuracy 77%. Large batch deep learning optimizer LARS for ImageNet with PyTorch and ResNet, using Horovod for distribution. Optional acc…
  ☆38Updated 3 years ago
• varunnair18 / FISH
  Code for "Training Neural Networks with Fixed Sparse Masks" (NeurIPS 2021).
  ☆56Updated 2 years ago
• hwang595 / Cuttlefish
  The implementation for MLSys 2023 paper: "Cuttlefish: Low-rank Model Training without All The Tuning"
  ☆43Updated last year
• pipilurj / BONAS
  ☆35Updated 3 years ago
• zyushun / hessian-spectrum
  Code for the paper: Why Transformers Need Adam: A Hessian Perspective
  ☆40Updated 6 months ago
• JingtongSu / sanity-checking-pruning
  Code for Sanity-Checking Pruning Methods: Random Tickets can Win the Jackpot
  ☆43Updated 4 years ago
• JeanKaddour / NoTrainNoGain
  Revisiting Efficient Training Algorithms For Transformer-based Language Models (NeurIPS 2023)
  ☆79Updated last year
• microsoft / Stochastic-Mixture-of-Experts
  This package implements THOR: Transformer with Stochastic Experts.
  ☆61Updated 3 years ago
• VITA-Group / Junk_DNA_Hypothesis
  [ICML 2024] Junk DNA Hypothesis: A Task-Centric Angle of LLM Pre-trained Weights through Sparsity; Lu Yin*, Ajay Jaiswal*, Shiwei Liu, So…
  ☆15Updated 5 months ago
• dguo98 / DiffPruning
  Parameter Efficient Transfer Learning with Diff Pruning
  ☆72Updated 3 years ago
• teelinsan / parallel-decoding
  Repository of the paper "Accelerating Transformer Inference for Translation via Parallel Decoding"
  ☆108Updated 7 months ago
• mit-han-lab / patch_conv
  Patch convolution to avoid large GPU memory usage of Conv2D
  ☆79Updated 5 months ago
• jianweif / OptimalGradCheckpointing
  ☆39Updated 3 years ago
• HazyResearch / fly
  ☆192Updated last year
• JonasGeiping / linear_cross_entropy_loss
  A fusion of a linear layer and a cross entropy loss, written for pytorch in triton.
  ☆54Updated 3 months ago
• shawntan / scattermoe
  Triton-based implementation of Sparse Mixture of Experts.
  ☆184Updated last month
• hahnyuan / ASVD4LLM
  Activation-aware Singular Value Decomposition for Compressing Large Language Models
  ☆49Updated 2 weeks ago
• VITA-Group / SMC-Bench
  [ICLR 2023] "Sparsity May Cry: Let Us Fail (Current) Sparse Neural Networks Together!" Shiwei Liu, Tianlong Chen, Zhenyu Zhang, Xuxi Chen…
  ☆27Updated last year
• juntang-zhuang / GSAM
  PyTorch repository for ICLR 2022 paper (GSAM) which improves generalization (e.g. +3.8% top-1 accuracy on ImageNet with ViT-B/32)
  ☆138Updated 2 years ago
• yxli2123 / LoSparse
  ☆46Updated last year
• zyxxmu / DSnoT
  Official Pytorch Implementation of Our Paper Accepted at ICLR 2024-- Dynamic Sparse No Training: Training-Free Fine-tuning for Sparse LLM…
  ☆35Updated 7 months ago
• Jiacheng-Zhu-AIML / AsymmetryLoRA
  Preprint: Asymmetry in Low-Rank Adapters of Foundation Models
  ☆29Updated 8 months ago
• UNITES-Lab / MC-SMoE
  [ICLR 2024 Spotlight] Code for the paper "Merge, Then Compress: Demystify Efficient SMoE with Hints from Its Routing Policy"
  ☆64Updated 5 months ago
• huggingface / block_movement_pruning
  Block Sparse movement pruning
  ☆78Updated 3 years ago
• tml-epfl / why-weight-decay
  Why Do We Need Weight Decay in Modern Deep Learning? [NeurIPS 2024]
  ☆49Updated last month
• IST-DASLab / WoodFisher
  Code accompanying the NeurIPS 2020 paper: WoodFisher (Singh & Alistarh, 2020)
  ☆46Updated 3 years ago
• RulinShao / LightSeq
  Official repository for LightSeq: Sequence Level Parallelism for Distributed Training of Long Context Transformers
  ☆196Updated 2 months ago
• DS3Lab / AC-SGD
  Code associated with the paper **Fine-tuning Language Models over Slow Networks using Activation Compression with Guarantees**.
  ☆25Updated last year
• locuslab / edge-of-stability
  ☆59Updated 3 years ago
• epfml / dynamic-sparse-flash-attention
  ☆132Updated last year