Alternatives and similar repositories for Moonlight:

Users that are interested in Moonlight are comparing it to the libraries listed below

• fla-org / native-sparse-attention
  🐳 Efficient Triton implementations for "Native Sparse Attention: Hardware-Aligned and Natively Trainable Sparse Attention"
  ☆590Updated last week
• Open-Reasoner-Zero / Open-Reasoner-Zero
  Official Repo for Open-Reasoner-Zero
  ☆1,667Updated 3 weeks ago
• MoonshotAI / MoBA
  MoBA: Mixture of Block Attention for Long-Context LLMs
  ☆1,687Updated 3 weeks ago
• PRIME-RL / PRIME
  Scalable RL solution for advanced reasoning of language models
  ☆1,419Updated last week
• SimpleBerry / LLaMA-O1
  Large Reasoning Models
  ☆800Updated 3 months ago
• ML-GSAI / LLaDA
  Official PyTorch implementation for "Large Language Diffusion Models"
  ☆1,313Updated 2 weeks ago
• AIDC-AI / Marco-o1
  An Open Large Reasoning Model for Real-World Solutions
  ☆1,475Updated 3 weeks ago
• BytedTsinghua-SIA / DAPO
  An Open-source RL System from ByteDance Seed and Tsinghua AIR
  ☆767Updated last week
• sail-sg / understand-r1-zero
  Understanding R1-Zero-Like Training: A Critical Perspective
  ☆568Updated this week
• KellerJordan / Muon
  Muon optimizer: +>30% sample efficiency with <3% wallclock overhead
  ☆529Updated this week
• allenai / OLMoE
  OLMoE: Open Mixture-of-Experts Language Models
  ☆693Updated 2 weeks ago
• seal-rg / recurrent-pretraining
  Pretraining code for a large-scale depth-recurrent language model
  ☆697Updated 2 weeks ago
• microsoft / MInference
  [NeurIPS'24 Spotlight, ICLR'25] To speed up Long-context LLMs' inference, approximate and dynamic sparse calculate the attention, which r…
  ☆945Updated last month
• agentica-project / deepscaler
  Democratizing Reinforcement Learning for LLMs
  ☆2,113Updated last month
• facebookresearch / memory
  Memory layers use a trainable key-value lookup mechanism to add extra parameters to a model without increasing FLOPs. Conceptually, spars…
  ☆310Updated 3 months ago
• GAIR-NLP / LIMO
  LIMO: Less is More for Reasoning
  ☆864Updated last month
• turningpoint-ai / VisualThinker-R1-Zero
  Explore the Multimodal “Aha Moment” on 2B Model
  ☆524Updated last week
• flashinfer-ai / flashinfer
  FlashInfer: Kernel Library for LLM Serving
  ☆2,483Updated this week
• Open-Source-O1 / Open-O1
  ☆1,348Updated 4 months ago
• zhuzilin / ring-flash-attention
  Ring attention implementation with flash attention
  ☆717Updated last month
• SafeAILab / EAGLE
  Official Implementation of EAGLE-1 (ICML'24), EAGLE-2 (EMNLP'24), and EAGLE-3.
  ☆1,094Updated this week
• facebookresearch / coconut
  Training Large Language Model to Reason in a Continuous Latent Space
  ☆998Updated 2 months ago
• baaivision / Emu3
  Next-Token Prediction is All You Need
  ☆2,042Updated last week
• zhentingqi / rStar
  ☆910Updated 2 months ago
• huggingface / nanotron
  Minimalistic large language model 3D-parallelism training
  ☆1,715Updated this week
• microsoft / rStar
  ☆485Updated last week
• jzhang38 / EasyContext
  Memory optimization and training recipes to extrapolate language models' context length to 1 million tokens, with minimal hardware.
  ☆706Updated 6 months ago
• deepseek-ai / DeepSeek-MoE
  DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Models
  ☆1,609Updated last year
• Qihoo360 / Light-R1
  ☆518Updated last week
• mlc-ai / xgrammar
  Fast, Flexible and Portable Structured Generation
  ☆818Updated this week