NVIDIA-NeMo / Megatron-BridgeLinks
HuggingFace conversion and training library for Megatron-based models
☆314Updated this week
Alternatives and similar repositories for Megatron-Bridge
Users that are interested in Megatron-Bridge are comparing it to the libraries listed below
Sorting:
- Implementation for FP8/INT8 Rollout for RL training without performence drop.☆282Updated last month
- ByteCheckpoint: An Unified Checkpointing Library for LFMs☆256Updated 3 weeks ago
- Official repository for DistFlashAttn: Distributed Memory-efficient Attention for Long-context LLMs Training☆219Updated last year
- Bridge Megatron-Core to Hugging Face/Reinforcement Learning☆174Updated last week
- 🚀 Efficiently (pre)training foundation models with native PyTorch features, including FSDP for training and SDPA implementation of Flash…☆276Updated last month
- 🔥 A minimal training framework for scaling FLA models☆327Updated last month
- ☆133Updated 7 months ago
- Best practices for training DeepSeek, Mixtral, Qwen and other MoE models using Megatron Core.☆143Updated 2 weeks ago
- Pytorch Distributed native training library for LLMs/VLMs with OOTB Hugging Face support☆223Updated this week
- Efficient triton implementation of Native Sparse Attention.☆257Updated 7 months ago
- Super-Efficient RLHF Training of LLMs with Parameter Reallocation☆330Updated 8 months ago
- Accelerating MoE with IO and Tile-aware Optimizations☆469Updated last week
- Triton implementation of FlashAttention2 that adds Custom Masks.☆158Updated last year
- Megatron's multi-modal data loader☆299Updated 2 weeks ago
- Ring attention implementation with flash attention☆958Updated 3 months ago
- PyTorch bindings for CUTLASS grouped GEMM.☆177Updated 2 weeks ago
- [ICLR 2025] COAT: Compressing Optimizer States and Activation for Memory-Efficient FP8 Training☆256Updated 4 months ago
- Triton-based implementation of Sparse Mixture of Experts.☆259Updated 2 months ago
- SeerAttention: Learning Intrinsic Sparse Attention in Your LLMs☆181Updated 3 months ago
- USP: Unified (a.k.a. Hybrid, 2D) Sequence Parallel Attention for Long Context Transformers Model Training and Inference☆618Updated last week
- [NeurIPS 2025] Simple extension on vLLM to help you speed up reasoning model without training.☆215Updated 7 months ago
- A lightweight reinforcement learning framework that integrates seamlessly into your codebase, empowering developers to focus on algorithm…☆95Updated 4 months ago
- Code for paper: [ICLR2025 Oral] FlexPrefill: A Context-Aware Sparse Attention Mechanism for Efficient Long-Sequence Inference☆159Updated 2 months ago
- (best/better) practices of megatron on veRL and tuning guide☆113Updated 3 months ago
- Async pipelined version of Verl☆124Updated 8 months ago
- [ICML 2024] Quest: Query-Aware Sparsity for Efficient Long-Context LLM Inference☆361Updated 5 months ago
- ☆654Updated this week
- [ICLR 2025] DuoAttention: Efficient Long-Context LLM Inference with Retrieval and Streaming Heads☆515Updated 10 months ago
- 🔥 LLM-powered GPU kernel synthesis: Train models to convert PyTorch ops into optimized Triton kernels via SFT+RL. Multi-turn compilation…☆110Updated last month
- ☆297Updated 5 months ago