NVIDIA-NeMo / Megatron-BridgeLinks
Training library for Megatron-based models with bidirectional Hugging Face conversion capability
☆419Updated this week
Alternatives and similar repositories for Megatron-Bridge
Users that are interested in Megatron-Bridge are comparing it to the libraries listed below
Sorting:
- ByteCheckpoint: An Unified Checkpointing Library for LFMs☆269Updated last week
- Official repository for DistFlashAttn: Distributed Memory-efficient Attention for Long-context LLMs Training☆222Updated last year
- Bridge Megatron-Core to Hugging Face/Reinforcement Learning☆191Updated last week
- 🚀 Efficiently (pre)training foundation models with native PyTorch features, including FSDP for training and SDPA implementation of Flash…☆279Updated 2 months ago
- Megatron's multi-modal data loader☆315Updated last week
- Super-Efficient RLHF Training of LLMs with Parameter Reallocation☆331Updated 9 months ago
- Ring attention implementation with flash attention☆979Updated 4 months ago
- Pytorch Distributed native training library for LLMs/VLMs with OOTB Hugging Face support☆266Updated this week
- USP: Unified (a.k.a. Hybrid, 2D) Sequence Parallel Attention for Long Context Transformers Model Training and Inference☆641Updated 3 weeks ago
- Implementation for FP8/INT8 Rollout for RL training without performence drop.☆289Updated 3 months ago
- Best practices for training DeepSeek, Mixtral, Qwen and other MoE models using Megatron Core.☆161Updated 2 weeks ago
- PyTorch bindings for CUTLASS grouped GEMM.☆184Updated last month
- [ICLR 2025] COAT: Compressing Optimizer States and Activation for Memory-Efficient FP8 Training☆258Updated 6 months ago
- 🔥 A minimal training framework for scaling FLA models☆343Updated 2 months ago
- Accelerating MoE with IO and Tile-aware Optimizations☆569Updated 3 weeks ago
- Zero Bubble Pipeline Parallelism☆449Updated 9 months ago
- [ICLR 2025] DuoAttention: Efficient Long-Context LLM Inference with Retrieval and Streaming Heads☆524Updated 11 months ago
- [ICML 2024] Quest: Query-Aware Sparsity for Efficient Long-Context LLM Inference☆370Updated 6 months ago
- Efficient triton implementation of Native Sparse Attention.☆262Updated 8 months ago
- Miles is an enterprise-facing reinforcement learning framework for LLM and VLM post-training, forked from and co-evolving with slime.☆830Updated this week
- Code for paper: [ICLR2025 Oral] FlexPrefill: A Context-Aware Sparse Attention Mechanism for Efficient Long-Sequence Inference☆160Updated 3 months ago
- PyTorch bindings for CUTLASS grouped GEMM.☆142Updated 8 months ago
- Triton implementation of FlashAttention2 that adds Custom Masks.☆165Updated last year
- Triton-based implementation of Sparse Mixture of Experts.☆263Updated 4 months ago
- ☆303Updated 6 months ago
- Train speculative decoding models effortlessly and port them smoothly to SGLang serving.☆676Updated this week
- Spec-Bench: A Comprehensive Benchmark and Unified Evaluation Platform for Speculative Decoding (ACL 2024 Findings)☆363Updated 9 months ago
- ☆131Updated 8 months ago
- (best/better) practices of megatron on veRL and tuning guide☆127Updated 4 months ago
- SeerAttention: Learning Intrinsic Sparse Attention in Your LLMs☆192Updated 4 months ago