nil0x9 / flash-muonLinks
Flash-Muon: An Efficient Implementation of Muon Optimizer
☆195Updated 4 months ago
Alternatives and similar repositories for flash-muon
Users that are interested in flash-muon are comparing it to the libraries listed below
Sorting:
- ☆130Updated 4 months ago
- The evaluation framework for training-free sparse attention in LLMs☆101Updated last week
- Fast and memory-efficient exact attention☆71Updated 7 months ago
- ☆251Updated 4 months ago
- ☆93Updated 8 months ago
- 🔥 A minimal training framework for scaling FLA models☆266Updated last month
- An efficient implementation of the NSA (Native Sparse Attention) kernel☆119Updated 4 months ago
- Efficient triton implementation of Native Sparse Attention.☆238Updated 5 months ago
- Triton implementation of FlashAttention2 that adds Custom Masks.☆141Updated last year
- ☆145Updated 8 months ago
- Triton-based implementation of Sparse Mixture of Experts.☆246Updated 3 weeks ago
- Normalized Transformer (nGPT)☆192Updated 11 months ago
- Work in progress.☆74Updated 3 months ago
- Accelerate LLM preference tuning via prefix sharing with a single line of code☆46Updated 3 months ago
- Odysseus: Playground of LLM Sequence Parallelism☆78Updated last year
- The simplest implementation of recent Sparse Attention patterns for efficient LLM inference.☆92Updated 3 months ago
- ring-attention experiments☆154Updated last year
- ☆121Updated last year
- [ICLR 2025] Official PyTorch Implementation of Gated Delta Networks: Improving Mamba2 with Delta Rule☆331Updated last month
- [ICLR 2025] COAT: Compressing Optimizer States and Activation for Memory-Efficient FP8 Training☆242Updated 2 months ago
- Tree Attention: Topology-aware Decoding for Long-Context Attention on GPU clusters☆130Updated 10 months ago
- Load compute kernels from the Hub☆304Updated last week
- Code for studying the super weight in LLM☆120Updated 10 months ago
- ☆64Updated 7 months ago
- Muon fsdp 2☆44Updated 2 months ago
- Implementation for FP8/INT8 Rollout for RL training without performence drop.☆261Updated 3 weeks ago
- Repo for "LoLCATs: On Low-Rank Linearizing of Large Language Models"☆248Updated 8 months ago
- ☆83Updated last year
- Code for paper: [ICLR2025 Oral] FlexPrefill: A Context-Aware Sparse Attention Mechanism for Efficient Long-Sequence Inference☆147Updated last week
- Memory optimized Mixture of Experts☆68Updated 3 months ago