Alternatives and similar repositories for MomentumSMoE

Users that are interested in MomentumSMoE are comparing it to the libraries listed below

• pixeli99 / MixLN
  [ICLR 2025] Official Pytorch Implementation of "Mix-LN: Unleashing the Power of Deeper Layers by Combining Pre-LN and Post-LN" by Pengxia…
  ☆26Updated last month
• assafbk / DeciMamba
  DeciMamba: Exploring the Length Extrapolation Potential of Mamba (ICLR 2025)
  ☆30Updated 4 months ago
• haiquanlu / AlphaPruning
  [NeurIPS 2024] AlphaPruning: Using Heavy-Tailed Self Regularization Theory for Improved Layer-wise Pruning of Large Language Models
  ☆27Updated 2 months ago
• euclid-multimodal / Euclid
  ☆17Updated 7 months ago
• TianjinYellow / SPAM-Optimizer
  ☆35Updated 5 months ago
• MikeWangWZHL / dymu
  ☆19Updated 3 months ago
• matchten / LoRA-Models-for-SAEs
  Code for reproducing our paper "Low Rank Adapting Models for Sparse Autoencoder Features"
  ☆14Updated 5 months ago
• ablghtianyi / ICL_Modular_Arithmetic
  ☆19Updated 5 months ago
• sail-sg / Attention-Sink
  [ICLR 2025] When Attention Sink Emerges in Language Models: An Empirical View (Spotlight)
  ☆119Updated last month
• fangyuan-ksgk / selective-attention-transformer
  Unofficial Implementation of Selective Attention Transformer
  ☆17Updated 10 months ago
• tianyi-lab / R2-T2
  [ICML 2025] Code for "R2-T2: Re-Routing in Test-Time for Multimodal Mixture-of-Experts"
  ☆16Updated 5 months ago
• Luckfort / CD
  [COLING'25] Exploring Concept Depth: How Large Language Models Acquire Knowledge at Different Layers?
  ☆80Updated 7 months ago
• g-luo / vlm_cross_modal_reps
  Official PyTorch Implementation for Vision-Language Models Create Cross-Modal Task Representations, ICML 2025
  ☆30Updated 4 months ago
• DavidFanzz / SCMoE
  ☆26Updated last year
• open-compass / ProSA
  [EMNLP 2024 Findings] ProSA: Assessing and Understanding the Prompt Sensitivity of LLMs
  ☆29Updated 3 months ago
• yayayacc / MUR
  ☆45Updated last month
• sail-sg / AnytimeReasoner
  Optimizing Anytime Reasoning via Budget Relative Policy Optimization
  ☆44Updated last month
• zzwjames / FailureLLMUnlearning
  An official implementation of "Catastrophic Failure of LLM Unlearning via Quantization" (ICLR 2025)
  ☆29Updated 6 months ago
• luka-group / vlm-knowledge-conflict
  Code for paper "Unraveling Cross-Modality Knowledge Conflicts in Large Vision-Language Models."
  ☆45Updated 10 months ago
• harveyhuang18 / EMR_Merging
  [NeurIPS 2024 Spotlight] EMR-Merging: Tuning-Free High-Performance Model Merging
  ☆67Updated 6 months ago
• scitix / MEAP
  Mask-Enhanced Autoregressive Prediction: Pay Less Attention to Learn More
  ☆33Updated 3 months ago
• CASE-Lab-UMD / Router-Tuning-Mixture-of-Depths
  The open-source Mixture of Depths code and the official implementation of the paper "Router-Tuning: A Simple and Effective Approach for E…
  ☆15Updated 2 weeks ago
• juyongjiang / KaSA
  [ICLR'25] Code for KaSA, an official implementation of "KaSA: Knowledge-Aware Singular-Value Adaptation of Large Language Models"
  ☆20Updated 7 months ago
• facebookresearch / AbstentionBench
  A holistic benchmark for LLM abstention
  ☆48Updated last week
• kyegomez / Mixture-of-Depths
  Implementation of the paper: "Mixture-of-Depths: Dynamically allocating compute in transformer-based language models"
  ☆104Updated last week
• Yuheng2000 / Awesome-LoRA
  Awesome Low-Rank Adaptation
  ☆43Updated 3 weeks ago
• kamanphoebe / Look-into-MoEs
  [NAACL 2025] A Closer Look into Mixture-of-Experts in Large Language Models
  ☆54Updated 6 months ago
• ArminAzizi98 / LaMDA
  ☆15Updated 9 months ago
• locuslab / llava-token-compression
  ☆43Updated 9 months ago
• VijayLingam95 / SVFT
  ☆31Updated 6 months ago