Alternatives and similar repositories for SLTrain:

Users that are interested in SLTrain are comparing it to the libraries listed below

• VITA-Group / Junk_DNA_Hypothesis
  [ICML 2024] Junk DNA Hypothesis: A Task-Centric Angle of LLM Pre-trained Weights through Sparsity; Lu Yin*, Ajay Jaiswal*, Shiwei Liu, So…
  ☆16Updated 7 months ago
• UNITES-Lab / moe-quantization
  Official code for the paper "Examining Post-Training Quantization for Mixture-of-Experts: A Benchmark"
  ☆10Updated 7 months ago
• ZO-Bench / ZO-LLM
  [ICML 2024] Official code for the paper "Revisiting Zeroth-Order Optimization for Memory-Efficient LLM Fine-Tuning: A Benchmark ".
  ☆83Updated 7 months ago
• pilancilab / Riemannian_Preconditioned_LoRA
  source code for paper "Riemannian Preconditioned LoRA for Fine-Tuning Foundation Models"
  ☆21Updated 7 months ago
• zyushun / hessian-spectrum
  Code for the paper: Why Transformers Need Adam: A Hessian Perspective
  ☆48Updated 9 months ago
• yxli2123 / LoSparse
  ☆49Updated last year
• EnnengYang / RepresentationSurgery
  Representation Surgery for Multi-Task Model Merging. ICML, 2024.
  ☆35Updated 3 months ago
• cjyaras / deep-lora-transformers
  Compressible Dynamics in Deep Overparameterized Low-Rank Learning & Adaptation (ICML'24 Oral)
  ☆13Updated 6 months ago
• MarlonBecker / MSAM
  ☆18Updated last year
• zyxxmu / DSnoT
  Official Pytorch Implementation of Our Paper Accepted at ICLR 2024-- Dynamic Sparse No Training: Training-Free Fine-tuning for Sparse LLM…
  ☆38Updated 9 months ago
• EnnengYang / AdaMerging
  AdaMerging: Adaptive Model Merging for Multi-Task Learning. ICLR, 2024.
  ☆63Updated 3 months ago
• OPTML-Group / DeepZero
  [ICLR'24] "DeepZero: Scaling up Zeroth-Order Optimization for Deep Model Training" by Aochuan Chen*, Yimeng Zhang*, Jinghan Jia, James Di…
  ☆49Updated 3 months ago
• osehmathias / lisa
  LISA: Layerwise Importance Sampling for Memory-Efficient Large Language Model Fine-Tuning
  ☆24Updated 9 months ago
• YefanZhou / TempBalance
  [NeurIPS 2023 Spotlight] Temperature Balancing, Layer-wise Weight Analysis, and Neural Network Training
  ☆31Updated last year
• stan-anony / derivative_free_lora_rank
  ☆15Updated 11 months ago
• tanganke / weight-ensembling_MoE
  Code for paper "Merging Multi-Task Models via Weight-Ensembling Mixture of Experts"
  ☆18Updated 7 months ago
• jiahai-feng / binding-iclr
  ☆12Updated 10 months ago
• ycjing / Awesome-Model-Merging
  A curated list of Model Merging methods.
  ☆89Updated 4 months ago
• Model-GLUE / Model-GLUE
  ☆11Updated 5 months ago
• VITA-Group / SMC-Bench
  [ICLR 2023] "Sparsity May Cry: Let Us Fail (Current) Sparse Neural Networks Together!" Shiwei Liu, Tianlong Chen, Zhenyu Zhang, Xuxi Chen…
  ☆27Updated last year
• VITA-Group / Random-MoE-as-Dropout
  [ICLR 2023] "Sparse MoE as the New Dropout: Scaling Dense and Self-Slimmable Transformers" by Tianlong Chen*, Zhenyu Zhang*, Ajay Jaiswal…
  ☆48Updated last year
• xu-ji / information-bottleneck
  Deep Learning & Information Bottleneck
  ☆53Updated last year
• sail-sg / Cheating-LLM-Benchmarks
  [SafeGenAi @ NeurIPS 2024] Cheating Automatic LLM Benchmarks: Null Models Achieve High Win Rates
  ☆68Updated 3 months ago
• JeanKaddour / NoTrainNoGain
  Revisiting Efficient Training Algorithms For Transformer-based Language Models (NeurIPS 2023)
  ☆79Updated last year
• nik-dim / tall_masks
  Official repository of "Localizing Task Information for Improved Model Merging and Compression" [ICML 2024]
  ☆39Updated 3 months ago
• htqin / IR-QLoRA
  [ICML 2024 Oral] This project is the official implementation of our Accurate LoRA-Finetuning Quantization of LLMs via Information Retenti…
  ☆60Updated 9 months ago
• decoding-comp-trust / comp-trust
  Codebase for decoding compressed trust.
  ☆22Updated 8 months ago
• WeiHuang05 / Awesome_Large_Foundation_Model_Theory
  Welcome to the 'In Context Learning Theory' Reading Group
  ☆27Updated 2 months ago
• GATECH-EIC / Linearized-LLM
  [ICML 2024] When Linear Attention Meets Autoregressive Decoding: Towards More Effective and Efficient Linearized Large Language Models
  ☆28Updated 7 months ago
• Leiay / looped_transformer
  ☆22Updated last year