NVIDIA-NeMo / Megatron-BridgeLinks
HuggingFace conversion and training library for Megatron-based models
☆250Updated this week
Alternatives and similar repositories for Megatron-Bridge
Users that are interested in Megatron-Bridge are comparing it to the libraries listed below
Sorting:
- Implementation for FP8/INT8 Rollout for RL training without performence drop.☆279Updated last month
- Official repository for DistFlashAttn: Distributed Memory-efficient Attention for Long-context LLMs Training☆219Updated last year
- ByteCheckpoint: An Unified Checkpointing Library for LFMs☆256Updated this week
- Megatron's multi-modal data loader☆287Updated this week
- Pytorch Distributed native training library for LLMs/VLMs with OOTB Hugging Face support☆202Updated this week
- Bridge Megatron-Core to Hugging Face/Reinforcement Learning☆168Updated last week
- Best practices for training DeepSeek, Mixtral, Qwen and other MoE models using Megatron Core.☆133Updated last month
- [ICLR 2025] COAT: Compressing Optimizer States and Activation for Memory-Efficient FP8 Training☆251Updated 4 months ago
- 🚀 Efficiently (pre)training foundation models with native PyTorch features, including FSDP for training and SDPA implementation of Flash…☆271Updated 2 weeks ago
- PyTorch bindings for CUTLASS grouped GEMM.☆172Updated 2 months ago
- ☆132Updated 6 months ago
- Super-Efficient RLHF Training of LLMs with Parameter Reallocation☆327Updated 7 months ago
- ☆293Updated 5 months ago
- Async pipelined version of Verl☆125Updated 8 months ago
- Train speculative decoding models effortlessly and port them smoothly to SGLang serving.☆539Updated this week
- Triton-based implementation of Sparse Mixture of Experts.☆253Updated 2 months ago
- 🔥 A minimal training framework for scaling FLA models☆319Updated 3 weeks ago
- [ICML 2024] Quest: Query-Aware Sparsity for Efficient Long-Context LLM Inference☆356Updated 5 months ago
- Ring attention implementation with flash attention☆937Updated 3 months ago
- Triton implementation of FlashAttention2 that adds Custom Masks.☆151Updated last year
- Odysseus: Playground of LLM Sequence Parallelism☆78Updated last year
- PyTorch bindings for CUTLASS grouped GEMM.☆132Updated 6 months ago
- [ICLR 2025] DuoAttention: Efficient Long-Context LLM Inference with Retrieval and Streaming Heads☆510Updated 10 months ago
- Code for paper: [ICLR2025 Oral] FlexPrefill: A Context-Aware Sparse Attention Mechanism for Efficient Long-Sequence Inference☆156Updated last month
- SeerAttention: Learning Intrinsic Sparse Attention in Your LLMs☆176Updated 2 months ago
- Zero Bubble Pipeline Parallelism☆437Updated 7 months ago
- ☆564Updated this week
- (best/better) practices of megatron on veRL and tuning guide☆108Updated 2 months ago
- A lightweight reinforcement learning framework that integrates seamlessly into your codebase, empowering developers to focus on algorithm…☆91Updated 3 months ago
- ring-attention experiments☆160Updated last year