ByteDance-Seed / VeOmniLinks
VeOmni: Scaling any Modality Model Training to any Accelerators with PyTorch native Training Framework
☆927Updated this week
Alternatives and similar repositories for VeOmni
Users that are interested in VeOmni are comparing it to the libraries listed below
Sorting:
- USP: Unified (a.k.a. Hybrid, 2D) Sequence Parallel Attention for Long Context Transformers Model Training and Inference☆549Updated last month
- ☆400Updated 2 weeks ago
- Ring attention implementation with flash attention☆841Updated 3 weeks ago
- Official implementation of "Fast-dLLM: Training-free Acceleration of Diffusion LLM by Enabling KV Cache and Parallel Decoding"☆374Updated this week
- ☆199Updated 4 months ago
- ☆811Updated 2 months ago
- 🐳 Efficient Triton implementations for "Native Sparse Attention: Hardware-Aligned and Natively Trainable Sparse Attention"☆822Updated 5 months ago
- Super-Efficient RLHF Training of LLMs with Parameter Reallocation☆309Updated 4 months ago
- A Distributed Attention Towards Linear Scalability for Ultra-Long Context, Heterogeneous Data Training☆489Updated last week
- slime is a LLM post-training framework aiming for RL Scaling.☆1,420Updated this week
- Muon is Scalable for LLM Training☆1,281Updated 3 weeks ago
- [Preprint] On the Generalization of SFT: A Reinforcement Learning Perspective with Reward Rectification.☆361Updated last week
- An Efficient and User-Friendly Scaling Library for Reinforcement Learning with Large Language Models☆1,755Updated this week
- FlagScale is a large model toolkit based on open-sourced projects.☆346Updated last week
- ByteCheckpoint: An Unified Checkpointing Library for LFMs☆237Updated last month
- The official repo of One RL to See Them All: Visual Triple Unified Reinforcement Learning☆312Updated 2 months ago
- ☆755Updated last month
- Kimi-VL: Mixture-of-Experts Vision-Language Model for Multimodal Reasoning, Long-Context Understanding, and Strong Agent Capabilities☆1,050Updated last month
- [Survey] Next Token Prediction Towards Multimodal Intelligence: A Comprehensive Survey☆447Updated 7 months ago
- Long-RL: Scaling RL to Long Sequences☆592Updated this week
- [ICLR 2025] DuoAttention: Efficient Long-Context LLM Inference with Retrieval and Streaming Heads☆487Updated 6 months ago
- Explore the Multimodal “Aha Moment” on 2B Model☆607Updated 5 months ago
- InternEvo is an open-sourced lightweight training framework aims to support model pre-training without the need for extensive dependencie…☆404Updated this week
- SpargeAttention: A training-free sparse attention that can accelerate any model inference.☆688Updated last week
- Parallel Scaling Law for Language Model — Beyond Parameter and Inference Time Scaling☆432Updated 3 months ago
- Extend OpenRLHF to support LMM RL training for reproduction of DeepSeek-R1 on multimodal tasks.☆812Updated 3 months ago
- Towards Economical Inference: Enabling DeepSeek's Multi-Head Latent Attention in Any Transformer-based LLMs☆188Updated 2 months ago
- Fast inference from large lauguage models via speculative decoding☆807Updated last year
- ☆513Updated last month
- A flexible and efficient training framework for large-scale alignment tasks☆415Updated this week