Alternatives and similar repositories for SPAM-Optimizer

Users that are interested in SPAM-Optimizer are comparing it to the libraries listed below

• zqOuO / GWT
  ☆13Updated 11 months ago
• zhixuan-lin / forgetting-transformer
  [ICLR 2025 & COLM 2025] Official PyTorch implementation of the Forgetting Transformer and Adaptive Computation Pruning
  ☆134Updated last week
• nasosger / MuToR
  [NeurIPS '25] Multi-Token Prediction Needs Registers
  ☆26Updated 2 weeks ago
• fangyuan-ksgk / selective-attention-transformer
  Unofficial Implementation of Selective Attention Transformer
  ☆18Updated last year
• chuanyang-Zheng / DAPE
  The this is the official implementation of "DAPE: Data-Adaptive Positional Encoding for Length Extrapolation"
  ☆39Updated last year
• lmsdss / LayerNorm-Scaling
  Official Pytorch Implementation of "The Curse of Depth in Large Language Models" by Wenfang Sun, Xinyuan Song, Pengxiang Li, Lu Yin,Yefen…
  ☆64Updated 3 weeks ago
• abdelfattah-lab / TokenButler
  ☆26Updated last month
• deep-spin / adasplash
  AdaSplash: Adaptive Sparse Flash Attention (aka Flash Entmax Attention)
  ☆31Updated 2 months ago
• nbasyl / DoRA
  Official implementation of "DoRA: Weight-Decomposed Low-Rank Adaptation"
  ☆124Updated last year
• IST-DASLab / QuEST
  Work in progress.
  ☆75Updated last month
• thu-ml / ReMoE
  [ICLR2025] Codebase for "ReMoE: Fully Differentiable Mixture-of-Experts with ReLU Routing", built on Megatron-LM.
  ☆102Updated last year
• assafbk / DeciMamba
  DeciMamba: Exploring the Length Extrapolation Potential of Mamba (ICLR 2025)
  ☆32Updated 8 months ago
• htqin / IR-QLoRA
  [ICML 2024 Oral] This project is the official implementation of our Accurate LoRA-Finetuning Quantization of LLMs via Information Retenti…
  ☆67Updated last year
• Anonymous1252022 / Megatron-DeepSpeed
  ☆15Updated last year
• thu-ml / TetraJet-MXFP4Training
  Pytorch implementation of "Oscillation-Reduced MXFP4 Training for Vision Transformers" on DeiT Model Pre-training
  ☆34Updated 6 months ago
• horseee / dKV-Cache
  [NeurIPS'25] dKV-Cache: The Cache for Diffusion Language Models
  ☆126Updated 7 months ago
• kyegomez / Mixture-of-Depths
  Implementation of the paper: "Mixture-of-Depths: Dynamically allocating compute in transformer-based language models"
  ☆111Updated 3 weeks ago
• nil0x9 / flash-muon
  Flash-Muon: An Efficient Implementation of Muon Optimizer
  ☆223Updated 6 months ago
• andyjm3 / SLTrain
  SLTrain: a sparse plus low-rank approach for parameter and memory efficient pretraining (NeurIPS 2024)
  ☆38Updated last year
• ldery / Bonsai
  Code for "Everybody Prune Now: Structured Pruning of LLMs with only Forward Passes"
  ☆29Updated last year
• RobertCsordas / moeut
  ☆91Updated last year
• jzhang38 / LongMamba
  Some preliminary explorations of Mamba's context scaling.
  ☆218Updated last year
• sramshetty / mixture-of-depths
  An unofficial implementation of "Mixture-of-Depths: Dynamically allocating compute in transformer-based language models"
  ☆36Updated last year
• jongwooko / distillm-2
  Official PyTorch implementation of DistiLLM-2: A Contrastive Approach Boosts the Distillation of LLMs (ICML 2025 Oral)
  ☆54Updated 6 months ago
• g-luo / vlm_cross_modal_reps
  Official PyTorch Implementation for Vision-Language Models Create Cross-Modal Task Representations, ICML 2025
  ☆31Updated 7 months ago
• TsinghuaC3I / Fourier-Position-Embedding
  [ICML 2025] Fourier Position Embedding: Enhancing Attention’s Periodic Extension for Length Generalization
  ☆105Updated 6 months ago
• OpenMOSS / LongLLaDA
  [AAAI26] LongLLaDA: Unlocking Long Context Capabilities in Diffusion LLMs
  ☆48Updated 3 weeks ago
• piotrpiekos / MoSA
  User-friendly implementation of the Mixture-of-Sparse-Attention (MoSA). MoSA selects distinct tokens for each head with expert choice rou…
  ☆28Updated 7 months ago
• Infini-AI-Lab / Kinetics
  Kinetics: Rethinking Test-Time Scaling Laws
  ☆84Updated 5 months ago
• ylsung / rsq
  Code for "RSQ: Learning from Important Tokens Leads to Better Quantized LLMs"
  ☆20Updated 6 months ago