Alternatives and similar repositories for Skywork-MoE

Users that are interested in Skywork-MoE are comparing it to the libraries listed below

• JT-Ushio / MHA2MLA
  Towards Economical Inference: Enabling DeepSeek's Multi-Head Latent Attention in Any Transformer-based LLMs
  ☆188Updated 2 months ago
• YuchuanTian / RethinkTinyLM
  [ICML'24] The official implementation of “Rethinking Optimization and Architecture for Tiny Language Models”
  ☆123Updated 7 months ago
• IEIT-Yuan / Yuan2.0-M32
  Mixture-of-Experts (MoE) Language Model
  ☆189Updated 11 months ago
• SkyworkAI / skywork-o1-prm-inference
  ☆65Updated 8 months ago
• bigai-nlco / TokenSwift
  [ICML 2025] |TokenSwift: Lossless Acceleration of Ultra Long Sequence Generation
  ☆113Updated 3 months ago
• yyht / openrlhf_async_pipline
  ☆72Updated last week
• FreedomIntelligence / FastLLM
  Fast LLM Training CodeBase With dynamic strategy choosing [Deepspeed+Megatron+FlashAttention+CudaFusionKernel+Compiler];
  ☆41Updated last year
• GAIR-NLP / ReAlign
  Reformatted Alignment
  ☆113Updated 11 months ago
• thunlp / Ouroboros
  Ouroboros: Speculative Decoding with Large Model Enhanced Drafting (EMNLP 2024 main)
  ☆109Updated 5 months ago
• OpenSparseLLMs / Linear-MoE
  ☆117Updated 2 months ago
• jshuadvd / LongRoPE
  Implementation of the LongRoPE: Extending LLM Context Window Beyond 2 Million Tokens Paper
  ☆149Updated last year
• MoonshotAI / Kimi-Researcher
  ☆69Updated 2 months ago
• pangu-tech / pangu-ultra
  ☆67Updated 2 months ago
• 18907305772 / FuseAI
  FuseAI Project
  ☆87Updated 6 months ago
• InternLM / OREAL
  Exploring the Limit of Outcome Reward for Learning Mathematical Reasoning
  ☆188Updated 5 months ago
• inferflow / inferflow
  Inferflow is an efficient and highly configurable inference engine for large language models (LLMs).
  ☆246Updated last year
• KaihuaTang / Qwen-Tokenizer-Pruner
  Due to the huge vocaburary size (151,936) of Qwen models, the Embedding and LM Head weights are excessively heavy. Therefore, this projec…
  ☆24Updated last year
• infinigence / LVEval
  Repository of LV-Eval Benchmark
  ☆69Updated 11 months ago
• astramind-ai / Mixture-of-depths
  Unofficial implementation for the paper "Mixture-of-Depths: Dynamically allocating compute in transformer-based language models"
  ☆169Updated last year
• OpenLMLab / scaling-rope
  code for Scaling Laws of RoPE-based Extrapolation
  ☆73Updated last year
• thu-ml / low-bit-optimizers
  Low-bit optimizers for PyTorch
  ☆130Updated last year
• microsoft / RedStone
  The RedStone repository includes code for preparing extensive datasets used in training large language models.
  ☆137Updated last month
• rlite-project / RLite
  A lightweight reinforcement learning framework that integrates seamlessly into your codebase, empowering developers to focus on algorithm…
  ☆47Updated last week
• Glaciohound / LM-Infinite
  Implementation of NAACL 2024 Outstanding Paper "LM-Infinite: Simple On-the-Fly Length Generalization for Large Language Models"
  ☆148Updated 5 months ago
• nick7nlp / Counting-Stars
  Counting-Stars (★)
  ☆83Updated 2 months ago
• Open-Source-O1 / o1_Reasoning_Patterns_Study
  ☆103Updated 8 months ago
• LLM360 / MegaMath
  [COLM 2025] An Open Math Pre-trainng Dataset with 370B Tokens.
  ☆98Updated 4 months ago
• RUC-GSAI / YuLan-Mini
  A highly capable 2.4B lightweight LLM using only 1T pre-training data with all details.
  ☆208Updated 3 weeks ago
• GAIR-NLP / ProX
  [ICML 2025] Programming Every Example: Lifting Pre-training Data Quality Like Experts at Scale
  ☆257Updated last month
• Tongyi-Zhiwen / QwenLong-L1
  ☆290Updated 2 months ago