Alternatives and similar repositories for MoE-LPR

Users that are interested in MoE-LPR are comparing it to the libraries listed below

• VITA-Group / Random-MoE-as-Dropout
  [ICLR 2023] "Sparse MoE as the New Dropout: Scaling Dense and Self-Slimmable Transformers" by Tianlong Chen*, Zhenyu Zhang*, Ajay Jaiswal…
  ☆56Updated 2 years ago
• kamanphoebe / Look-into-MoEs
  [NAACL 2025] A Closer Look into Mixture-of-Experts in Large Language Models
  ☆56Updated 10 months ago
• ldery / Bonsai
  Code for "Everybody Prune Now: Structured Pruning of LLMs with only Forward Passes"
  ☆29Updated last year
• shaochenze / PatchTrain
  Code for paper "Patch-Level Training for Large Language Models"
  ☆96Updated last month
• Infini-AI-Lab / S2FT
  ☆19Updated 11 months ago
• Leooyii / LCEG
  Long Context Extension and Generalization in LLMs
  ☆62Updated last year
• song-wx / SIFT
  [ICML2024 Spotlight] Fine-Tuning Pre-trained Large Language Models Sparsely
  ☆24Updated last year
• Shwai-He / MEO
  The source code of "Merging Experts into One: Improving Computational Efficiency of Mixture of Experts (EMNLP 2023)":
  ☆44Updated last year
• DRSY / KV_Compression
  [EMNLP 2023]Context Compression for Auto-regressive Transformers with Sentinel Tokens
  ☆25Updated 2 years ago
• htqin / IR-QLoRA
  [ICML 2024 Oral] This project is the official implementation of our Accurate LoRA-Finetuning Quantization of LLMs via Information Retenti…
  ☆67Updated last year
• thu-coai / MiniPLM
  [ICLR 2025] MiniPLM: Knowledge Distillation for Pre-Training Language Models
  ☆69Updated last year
• GATECH-EIC / Linearized-LLM
  [ICML 2024] When Linear Attention Meets Autoregressive Decoding: Towards More Effective and Efficient Linearized Large Language Models
  ☆35Updated last year
• OpenSparseLLMs / MoM
  ☆114Updated 3 months ago
• MikaStars39 / StableMask
  PyTorch implementation of StableMask (ICML'24)
  ☆14Updated last year
• raymin0223 / fast_robust_early_exit
  Fast and Robust Early-Exiting Framework for Autoregressive Language Models with Synchronized Parallel Decoding (EMNLP 2023 Long)
  ☆65Updated last year
• xichen-fy / Fira
  Fira: Can We Achieve Full-rank Training of LLMs Under Low-rank Constraint?
  ☆119Updated last year
• thunlp / SparsingLaw
  The open-source materials for paper "Sparsing Law: Towards Large Language Models with Greater Activation Sparsity".
  ☆28Updated last year
• PKU-ML / LongPPL
  Code for ICLR 2025 Paper "What is Wrong with Perplexity for Long-context Language Modeling?"
  ☆107Updated 2 months ago
• BestAnHongjun / SentenceVAE
  Enable Next-sentence Prediction for Large Language Models with Faster Speed, Higher Accuracy and Longer Context
  ☆41Updated last year
• TsinghuaC3I / Fourier-Position-Embedding
  [ICML 2025] Fourier Position Embedding: Enhancing Attention’s Periodic Extension for Length Generalization
  ☆105Updated 6 months ago
• abdelfattah-lab / TokenButler
  ☆26Updated last month
• chuanyang-Zheng / DAPE
  The this is the official implementation of "DAPE: Data-Adaptive Positional Encoding for Length Extrapolation"
  ☆39Updated last year
• OpenSparseLLMs / Linear-MoE
  ☆126Updated 6 months ago
• YuchuanTian / RethinkTinyLM
  [ICML'24] The official implementation of “Rethinking Optimization and Architecture for Tiny Language Models”
  ☆126Updated 11 months ago
• thunlp / MoEfication
  ☆142Updated last year
• YuchuanTian / DiJiang
  [ICML'24 Oral] The official code of "DiJiang: Efficient Large Language Models through Compact Kernelization", a novel DCT-based linear at…
  ☆104Updated last year
• Infini-AI-Lab / gsm_infinite
  ☆60Updated 6 months ago
• OpenSparseLLMs / Open-Pandora
  Open-Pandora: On-the-fly Control Video Generation
  ☆35Updated last year
• VITA-Group / WeLore
  [ICML 2025] From Low Rank Gradient Subspace Stabilization to Low-Rank Weights: Observations, Theories and Applications
  ☆52Updated last month
• VILA-Lab / GBLM-Pruner
  Are gradient information useful for pruning of LLMs?
  ☆47Updated 4 months ago