Alternatives and similar repositories for Moonlight:

Users that are interested in Moonlight are comparing it to the libraries listed below

• MoonshotAI / MoBA
  MoBA: Mixture of Block Attention for Long-Context LLMs
  ☆1,746Updated 2 weeks ago
• Open-Reasoner-Zero / Open-Reasoner-Zero
  Official Repo for Open-Reasoner-Zero
  ☆1,872Updated last week
• ByteDance-Seed / Seed-Thinking-v1.5
  ☆662Updated this week
• PRIME-RL / PRIME
  Scalable RL solution for advanced reasoning of language models
  ☆1,488Updated last month
• sail-sg / understand-r1-zero
  Understanding R1-Zero-Like Training: A Critical Perspective
  ☆863Updated this week
• fla-org / native-sparse-attention
  🐳 Efficient Triton implementations for "Native Sparse Attention: Hardware-Aligned and Natively Trainable Sparse Attention"
  ☆631Updated last month
• allenai / OLMoE
  OLMoE: Open Mixture-of-Experts Language Models
  ☆713Updated last month
• SimpleBerry / LLaMA-O1
  Large Reasoning Models
  ☆802Updated 4 months ago
• HKUNLP / Dream
  Dream 7B, a large diffusion language model
  ☆551Updated last week
• microsoft / MInference
  [NeurIPS'24 Spotlight, ICLR'25] To speed up Long-context LLMs' inference, approximate and dynamic sparse calculate the attention, which r…
  ☆971Updated this week
• huggingface / search-and-learn
  Recipes to scale inference-time compute of open models
  ☆1,055Updated last month
• MoonshotAI / Kimi-VL
  Kimi-VL: Mixture-of-Experts Vision-Language Model for Multimodal Reasoning, Long-Context Understanding, and Strong Agent Capabilities
  ☆685Updated this week
• KellerJordan / Muon
  Muon optimizer: +>30% sample efficiency with <3% wallclock overhead
  ☆575Updated 3 weeks ago
• BytedTsinghua-SIA / DAPO
  An Open-source RL System from ByteDance Seed and Tsinghua AIR
  ☆1,141Updated last week
• seal-rg / recurrent-pretraining
  Pretraining code for a large-scale depth-recurrent language model
  ☆743Updated last week
• lucidrains / native-sparse-attention-pytorch
  Implementation of the sparse attention pattern proposed by the Deepseek team in their "Native Sparse Attention" paper
  ☆593Updated 3 weeks ago
• ML-GSAI / LLaDA
  Official PyTorch implementation for "Large Language Diffusion Models"
  ☆1,492Updated 2 weeks ago
• GAIR-NLP / LIMO
  LIMO: Less is More for Reasoning
  ☆913Updated 2 weeks ago
• huggingface / picotron
  Minimalistic 4D-parallelism distributed training framework for education purpose
  ☆987Updated last month
• facebookresearch / memory
  Memory layers use a trainable key-value lookup mechanism to add extra parameters to a model without increasing FLOPs. Conceptually, spars…
  ☆317Updated 4 months ago
• jzhang38 / EasyContext
  Memory optimization and training recipes to extrapolate language models' context length to 1 million tokens, with minimal hardware.
  ☆715Updated 6 months ago
• AIDC-AI / Marco-o1
  An Open Large Reasoning Model for Real-World Solutions
  ☆1,484Updated last month
• SkyworkAI / Skywork-OR1
  Unleashing the Power of Reinforcement Learning for Math and Code Reasoners
  ☆490Updated this week
• flashinfer-ai / flashinfer
  FlashInfer: Kernel Library for LLM Serving
  ☆2,659Updated last week
• turningpoint-ai / VisualThinker-R1-Zero
  Explore the Multimodal “Aha Moment” on 2B Model
  ☆572Updated last month
• RAGEN-AI / RAGEN
  RAGEN leverages reinforcement learning to train LLM reasoning agents in interactive, stochastic environments.
  ☆1,382Updated this week
• SafeAILab / EAGLE
  Official Implementation of EAGLE-1 (ICML'24), EAGLE-2 (EMNLP'24), and EAGLE-3.
  ☆1,183Updated this week
• huggingface / Math-Verify
  ☆630Updated 3 weeks ago
• huggingface / nanotron
  Minimalistic large language model 3D-parallelism training
  ☆1,793Updated this week
• zhuzilin / ring-flash-attention
  Ring attention implementation with flash attention
  ☆737Updated last week