Alternatives and similar repositories for Skywork-MoE:

Users that are interested in Skywork-MoE are comparing it to the libraries listed below

• IEIT-Yuan / Yuan2.0-M32
  Mixture-of-Experts (MoE) Language Model
  ☆184Updated 5 months ago
• YuchuanTian / RethinkTinyLM
  [ICML'24] The official implementation of “Rethinking Optimization and Architecture for Tiny Language Models”
  ☆120Updated last month
• jshuadvd / LongRoPE
  Implementation of the LongRoPE: Extending LLM Context Window Beyond 2 Million Tokens Paper
  ☆125Updated 7 months ago
• SkyworkAI / skywork-o1-prm-inference
  ☆57Updated 2 months ago
• GAIR-NLP / ProX
  Offical Repo for "Programming Every Example: Lifting Pre-training Data Quality Like Experts at Scale"
  ☆216Updated this week
• GAIR-NLP / ReAlign
  Reformatted Alignment
  ☆114Updated 4 months ago
• THUDM / LongAlign
  [EMNLP 2024] LongAlign: A Recipe for Long Context Alignment of LLMs
  ☆241Updated 2 months ago
• TemporaryLoRA / Temp-LoRA
  ☆96Updated 10 months ago
• OpenLMLab / scaling-rope
  code for Scaling Laws of RoPE-based Extrapolation
  ☆70Updated last year
• thunlp / Ouroboros
  Ouroboros: Speculative Decoding with Large Model Enhanced Drafting (EMNLP 2024 main)
  ☆84Updated 4 months ago
• astramind-ai / Mixture-of-depths
  Unofficial implementation for the paper "Mixture-of-Depths: Dynamically allocating compute in transformer-based language models"
  ☆145Updated 8 months ago
• SparkJiao / llama-pipeline-parallel
  A prototype repo for hybrid training of pipeline parallel and distributed data parallel with comments on core code snippets. Feel free to…
  ☆55Updated last year
• inferflow / inferflow
  Inferflow is an efficient and highly configurable inference engine for large language models (LLMs).
  ☆238Updated 11 months ago
• zhaochenyang20 / ModelServer
  Efficient, Flexible, and Highly Fault-Tolerant Model Service Management Based on SGLang
  ☆35Updated 3 months ago
• thu-ml / low-bit-optimizers
  Low-bit optimizers for PyTorch
  ☆125Updated last year
• Bui1dMySea / MemLong
  ☆91Updated 2 months ago
• infinigence / LVEval
  Repository of LV-Eval Benchmark
  ☆58Updated 5 months ago
• thunlp / Delta-CoMe
  Delta-CoMe can achieve near loss-less 1-bit compressin which has been accepted by NeurIPS 2024
  ☆53Updated 3 months ago
• SomeoneKong / llm_long_context_bench202405
  ☆28Updated 5 months ago
• FreedomIntelligence / FastLLM
  Fast LLM Training CodeBase With dynamic strategy choosing [Deepspeed+Megatron+FlashAttention+CudaFusionKernel+Compiler];
  ☆36Updated last year
• alibaba / ChatLearn
  A flexible and efficient training framework for large-scale alignment tasks
  ☆303Updated last week
• HarderThenHarder / RLLoggingBoard
  A visuailzation tool to make deep understaning and easier debugging for RLHF training.
  ☆145Updated last month
• Wangmerlyn / MCTS-GSM8k-Demo
  This is a repo for showcasing using MCTS with LLMs to solve gsm8k problems
  ☆49Updated last month
• THUDM / ChatGLM-Math
  ☆81Updated 10 months ago
• Infini-AI-Lab / MagicDec
  [ICLR2025] Breaking Throughput-Latency Trade-off for Long Sequences with Speculative Decoding
  ☆107Updated 2 months ago
• zms1999 / SmartMoE
  A MoE impl for PyTorch, [ATC'23] SmartMoE
  ☆61Updated last year