Alternatives and similar repositories for CoE

Users that are interested in CoE are comparing it to the libraries listed below

• cmu-l3 / l1
  L1: Controlling How Long A Reasoning Model Thinks With Reinforcement Learning
  ☆198Updated last week
• JT-Ushio / MHA2MLA
  Towards Economical Inference: Enabling DeepSeek's Multi-Head Latent Attention in Any Transformer-based LLMs
  ☆164Updated last week
• XunhaoLai / native-sparse-attention-triton
  Efficient triton implementation of Native Sparse Attention.
  ☆144Updated last month
• Tencent / llm.hunyuan.T1
  ☆76Updated last month
• CMU-AIRe / MRT
  Research Code for preprint "Optimizing Test-Time Compute via Meta Reinforcement Finetuning".
  ☆93Updated 2 months ago
• itsnamgyu / block-transformer
  Block Transformer: Global-to-Local Language Modeling for Fast Inference (NeurIPS 2024)
  ☆154Updated last month
• TIGER-AI-Lab / CritiqueFineTuning
  Code for "Critique Fine-Tuning: Learning to Critique is More Effective than Learning to Imitate"
  ☆144Updated 3 weeks ago
• hao-ai-lab / Dynasor
  Simple extension on vLLM to help you speed up reasoning model without training.
  ☆149Updated last week
• OpenSparseLLMs / Linear-MoE
  ☆82Updated this week
• efficientscaling / Z1
  Repo for "Z1: Efficient Test-time Scaling with Code"
  ☆58Updated last month
• LLM360 / MegaMath
  An Open Math Pre-trainng Dataset with 370B Tokens.
  ☆80Updated last month
• SkyworkAI / Skywork-MoE
  Skywork-MoE: A Deep Dive into Training Techniques for Mixture-of-Experts Language Models
  ☆132Updated 11 months ago
• hemingkx / TokenSkip
  TokenSkip: Controllable Chain-of-Thought Compression in LLMs
  ☆138Updated 2 months ago
• RUC-GSAI / YuLan-Mini
  A highly capable 2.4B lightweight LLM using only 1T pre-training data with all details.
  ☆177Updated last month
• GAIR-NLP / LIMR
  ☆194Updated 2 months ago
• astramind-ai / Mixture-of-depths
  Unofficial implementation for the paper "Mixture-of-Depths: Dynamically allocating compute in transformer-based language models"
  ☆158Updated 10 months ago
• SalesforceAIResearch / GemFilter
  ☆78Updated 3 months ago
• facebookresearch / sweet_rl
  Benchmark and research code for the paper SWEET-RL Training Multi-Turn LLM Agents onCollaborative Reasoning Tasks
  ☆188Updated last week
• NovaSky-AI / SkyRL
  SkyRL-v0: Train Real-World Long-Horizon Agents via Reinforcement Learning
  ☆180Updated this week
• lzhxmu / CPPO
  CPPO: Accelerating the Training of Group Relative Policy Optimization-Based Reasoning Models
  ☆125Updated last week
• hkust-nlp / PreSelect
  ☆46Updated 2 months ago
• LCLM-Horizon / A-Comprehensive-Survey-For-Long-Context-Language-Modeling
  A Comprehensive Survey on Long Context Language Modeling
  ☆139Updated last month
• ypwang61 / One-Shot-RLVR
  official repository for “Reinforcement Learning for Reasoning in Large Language Models with One Training Example”
  ☆143Updated last week
• HKUDS / SepLLM
  SepLLM: Accelerate Large Language Models by Compressing One Segment into One Separator
  ☆71Updated 4 months ago
• GAIR-NLP / ToRL
  ☆153Updated last month
• lucidrains / coconut-pytorch
  Implementation of 🥥 Coconut, Chain of Continuous Thought, in Pytorch
  ☆166Updated 4 months ago
• tianyi-lab / MoE-Embedding
  Code for "Your Mixture-of-Experts LLM Is Secretly an Embedding Model For Free"
  ☆68Updated 6 months ago
• fla-org / flame
  🔥 A minimal training framework for scaling FLA models
  ☆128Updated this week
• kyegomez / Mixture-of-Depths
  Implementation of the paper: "Mixture-of-Depths: Dynamically allocating compute in transformer-based language models"
  ☆91Updated this week
• facebookresearch / memory
  Memory layers use a trainable key-value lookup mechanism to add extra parameters to a model without increasing FLOPs. Conceptually, spars…
  ☆323Updated 5 months ago