Alternatives and similar repositories for Skywork-MoE

Users that are interested in Skywork-MoE are comparing it to the libraries listed below

• JT-Ushio / MHA2MLA
  Towards Economical Inference: Enabling DeepSeek's Multi-Head Latent Attention in Any Transformer-based LLMs
  ☆166Updated last week
• infinigence / LVEval
  Repository of LV-Eval Benchmark
  ☆65Updated 9 months ago
• YuchuanTian / RethinkTinyLM
  [ICML'24] The official implementation of “Rethinking Optimization and Architecture for Tiny Language Models”
  ☆121Updated 4 months ago
• IEIT-Yuan / Yuan2.0-M32
  Mixture-of-Experts (MoE) Language Model
  ☆188Updated 8 months ago
• Tongyi-Zhiwen / QwenLong-L1
  ☆131Updated this week
• SkyworkAI / skywork-o1-prm-inference
  ☆63Updated 6 months ago
• zms1999 / SmartMoE
  A MoE impl for PyTorch, [ATC'23] SmartMoE
  ☆62Updated last year
• bigai-nlco / TokenSwift
  [ICML 2025] |TokenSwift: Lossless Acceleration of Ultra Long Sequence Generation
  ☆101Updated 2 weeks ago
• 18907305772 / FuseAI
  FuseAI Project
  ☆87Updated 4 months ago
• yyht / openrlhf_async_pipline
  ☆48Updated 3 weeks ago
• LCLM-Horizon / A-Comprehensive-Survey-For-Long-Context-Language-Modeling
  A Comprehensive Survey on Long Context Language Modeling
  ☆147Updated last week
• thu-ml / low-bit-optimizers
  Low-bit optimizers for PyTorch
  ☆128Updated last year
• InternLM / OREAL
  Exploring the Limit of Outcome Reward for Learning Mathematical Reasoning
  ☆179Updated 2 months ago
• GAIR-NLP / ProX
  [ICML 2025] Programming Every Example: Lifting Pre-training Data Quality Like Experts at Scale
  ☆248Updated 2 weeks ago
• jshuadvd / LongRoPE
  Implementation of the LongRoPE: Extending LLM Context Window Beyond 2 Million Tokens Paper
  ☆136Updated 10 months ago
• Bui1dMySea / MemLong
  ☆94Updated 5 months ago
• LLM360 / MegaMath
  An Open Math Pre-trainng Dataset with 370B Tokens.
  ☆87Updated last month
• GAIR-NLP / ReAlign
  Reformatted Alignment
  ☆113Updated 8 months ago
• modelscope / Trinity-RFT
  Trinity-RFT is a general-purpose, flexible and scalable framework designed for reinforcement fine-tuning (RFT) of large language models (…
  ☆100Updated this week
• THUDM / LongAlign
  [EMNLP 2024] LongAlign: A Recipe for Long Context Alignment of LLMs
  ☆249Updated 5 months ago
• SomeoneKong / llm_long_context_bench202405
  ☆29Updated 9 months ago
• TemporaryLoRA / Temp-LoRA
  ☆104Updated last year
• thunlp / Ouroboros
  Ouroboros: Speculative Decoding with Large Model Enhanced Drafting (EMNLP 2024 main)
  ☆106Updated 2 months ago
• Tencent / llm.hunyuan.T1
  ☆77Updated last month
• princeton-nlp / CEPE
  [ACL 2024] Long-Context Language Modeling with Parallel Encodings
  ☆153Updated 11 months ago
• jiahe7ay / infini-mini-transformer
  This is a personal reimplementation of Google's Infini-transformer, utilizing a small 2b model. The project includes both model and train…
  ☆56Updated last year
• GAIR-NLP / LIMR
  ☆201Updated 3 months ago
• RUC-GSAI / YuLan-Mini
  A highly capable 2.4B lightweight LLM using only 1T pre-training data with all details.
  ☆182Updated this week
• inferflow / inferflow
  Inferflow is an efficient and highly configurable inference engine for large language models (LLMs).
  ☆242Updated last year
• FlagOpen / Infinity-Instruct
  ☆47Updated 11 months ago