MoonshotAI / Moonlight
Muon is Scalable for LLM Training
☆1,043Updated last month
Alternatives and similar repositories for Moonlight:
Users that are interested in Moonlight are comparing it to the libraries listed below
- MoBA: Mixture of Block Attention for Long-Context LLMs☆1,771Updated last month
- 🐳 Efficient Triton implementations for "Native Sparse Attention: Hardware-Aligned and Natively Trainable Sparse Attention"☆653Updated last month
- Scalable RL solution for advanced reasoning of language models☆1,537Updated last month
- Official Repo for Open-Reasoner-Zero☆1,912Updated last month
- Recipes to scale inference-time compute of open models☆1,068Updated this week
- Dream 7B, a large diffusion language model☆622Updated last week
- ☆739Updated 3 weeks ago
- Understanding R1-Zero-Like Training: A Critical Perspective☆915Updated 3 weeks ago
- Large Reasoning Models☆804Updated 5 months ago
- Muon optimizer: +>30% sample efficiency with <3% wallclock overhead☆611Updated last month
- Kimi-VL: Mixture-of-Experts Vision-Language Model for Multimodal Reasoning, Long-Context Understanding, and Strong Agent Capabilities☆829Updated 3 weeks ago
- An Open Large Reasoning Model for Real-World Solutions☆1,488Updated 2 months ago
- An Open-source RL System from ByteDance Seed and Tsinghua AIR☆1,219Updated last month
- ☆683Updated last week
- Minimalistic large language model 3D-parallelism training☆1,850Updated this week
- Training Large Language Model to Reason in a Continuous Latent Space☆1,104Updated 3 months ago
- O1 Replication Journey☆1,987Updated 3 months ago
- RAGEN leverages reinforcement learning to train LLM reasoning agents in interactive, stochastic environments.☆1,772Updated this week
- OLMoE: Open Mixture-of-Experts Language Models☆739Updated last month
- LIMO: Less is More for Reasoning☆933Updated last month
- Implementation of the sparse attention pattern proposed by the Deepseek team in their "Native Sparse Attention" paper☆613Updated last month
- OpenR: An Open Source Framework for Advanced Reasoning with Large Language Models☆1,765Updated 3 months ago
- ☆928Updated 3 months ago
- [NeurIPS'24 Spotlight, ICLR'25, ICML'25] To speed up Long-context LLMs' inference, approximate and dynamic sparse calculate the attention…☆1,013Updated last week
- EasyR1: An Efficient, Scalable, Multi-Modality RL Training Framework based on veRL☆2,315Updated this week
- ☆684Updated 3 weeks ago
- Fast, Flexible and Portable Structured Generation☆922Updated this week
- Memory optimization and training recipes to extrapolate language models' context length to 1 million tokens, with minimal hardware.☆723Updated 7 months ago
- Simple RL training for reasoning☆3,540Updated last month
- A fork to add multimodal model training to open-r1☆1,252Updated 3 months ago