Alternatives and similar repositories for hessian-spectrum

Users that are interested in hessian-spectrum are comparing it to the libraries listed below

• ZO-Bench / ZO-LLM
  [ICML 2024] Official code for the paper "Revisiting Zeroth-Order Optimization for Memory-Efficient LLM Fine-Tuning: A Benchmark ".
  ☆104Updated 11 months ago
• locuslab / edge-of-stability
  ☆67Updated 5 months ago
• osehmathias / lisa
  LISA: Layerwise Importance Sampling for Memory-Efficient Large Language Model Fine-Tuning
  ☆31Updated last year
• andyjm3 / SLTrain
  SLTrain: a sparse plus low-rank approach for parameter and memory efficient pretraining (NeurIPS 2024)
  ☆31Updated 7 months ago
• aim-uofa / LoRAPrune
  ☆54Updated 5 months ago
• YefanZhou / TempBalance
  [NeurIPS 2023 Spotlight] Temperature Balancing, Layer-wise Weight Analysis, and Neural Network Training
  ☆35Updated last month
• OpenSparseLLMs / MoM
  ☆83Updated last month
• shawntan / stickbreaking-attention
  Stick-breaking attention
  ☆56Updated 2 months ago
• zhixuan-lin / forgetting-transformer
  [ICLR 2025] Official PyTorch implementation of "Forgetting Transformer: Softmax Attention with a Forget Gate"
  ☆104Updated 3 weeks ago
• TianjinYellow / SPAM-Optimizer
  ☆29Updated 2 months ago
• Jiacheng-Zhu-AIML / AsymmetryLoRA
  Preprint: Asymmetry in Low-Rank Adapters of Foundation Models
  ☆35Updated last year
• JeanKaddour / NoTrainNoGain
  Revisiting Efficient Training Algorithms For Transformer-based Language Models (NeurIPS 2023)
  ☆80Updated last year
• tml-epfl / why-weight-decay
  Why Do We Need Weight Decay in Modern Deep Learning? [NeurIPS 2024]
  ☆66Updated 8 months ago
• PKU-ML / LongPPL
  Code for ICLR 2025 Paper "What is Wrong with Perplexity for Long-context Language Modeling?"
  ☆78Updated 3 weeks ago
• hanyang1999 / discrete-diffusion-papers
  A collection of papers on discrete diffusion models
  ☆121Updated last week
• sustcsonglin / linear-attention-and-beyond-slides
  ☆74Updated 3 months ago
• holarissun / RewardModelingBeyondBradleyTerry
  official implementation of ICLR'2025 paper: Rethinking Bradley-Terry Models in Preference-based Reward Modeling: Foundations, Theory, and…
  ☆58Updated 2 months ago
• alvin-zyl / CoLA
  Implementation of CoLA: Compute-Efficient Pre-Training of LLMs via Low-Rank Activation
  ☆22Updated 3 months ago
• locuslab / massive-activations
  Code accompanying the paper "Massive Activations in Large Language Models"
  ☆162Updated last year
• thu-ml / ReMoE
  [ICLR2025] Codebase for "ReMoE: Fully Differentiable Mixture-of-Experts with ReLU Routing", built on Megatron-LM.
  ☆74Updated 5 months ago
• fla-org / flame
  🔥 A minimal training framework for scaling FLA models
  ☆146Updated 3 weeks ago
• facebookresearch / iGSM
  The code for creating the iGSM datasets in papers "Physics of Language Models Part 2.1, Grade-School Math and the Hidden Reasoning Proces…
  ☆54Updated 4 months ago
• allenbai01 / transformers-as-statisticians
  ☆32Updated last year
• VITA-Group / SMC-Bench
  [ICLR 2023] "Sparsity May Cry: Let Us Fail (Current) Sparse Neural Networks Together!" Shiwei Liu, Tianlong Chen, Zhenyu Zhang, Xuxi Chen…
  ☆28Updated last year
• berlino / seq_icl
  ☆52Updated last year
• WeiHuang05 / Awesome_Large_Foundation_Model_Theory
  Welcome to the 'In Context Learning Theory' Reading Group
  ☆28Updated 6 months ago
• sail-sg / win
  ☆9Updated last year
• Qualcomm-AI-research / llm-surgeon
  ☆29Updated last year
• yuezhouhu / 2by4-pretrain
  Efficient 2:4 sparse training algorithms and implementations
  ☆54Updated 5 months ago
• ryoungj / BoLT
  Code for "Reasoning to Learn from Latent Thoughts"
  ☆104Updated 2 months ago