liangyuwang / Tiny-MegatronLinks
Tiny-Megatron, a minimalistic re-implementation of the Megatron library
☆17Updated 3 weeks ago
Alternatives and similar repositories for Tiny-Megatron
Users that are interested in Tiny-Megatron are comparing it to the libraries listed below
Sorting:
- A Survey of Efficient Attention Methods: Hardware-efficient, Sparse, Compact, and Linear Attention☆181Updated last month
- [NeurIPS 2024] Efficient LLM Scheduling by Learning to Rank☆59Updated 10 months ago
- analyse problems of AI with Math and Code☆24Updated last month
- DeepSeek Native Sparse Attention pytorch implementation☆96Updated last month
- ☆42Updated last year
- ☆143Updated 2 months ago
- [ICLR 2025] PEARL: Parallel Speculative Decoding with Adaptive Draft Length☆113Updated 5 months ago
- [DAC'25] Official implement of "HybriMoE: Hybrid CPU-GPU Scheduling and Cache Management for Efficient MoE Inference"☆71Updated 3 months ago
- siiRL: Shanghai Innovation Institute RL Framework for Advanced LLMs and Multi-Agent Systems☆214Updated this week
- ☆50Updated 4 months ago
- Implementation for FP8/INT8 Rollout for RL training without performence drop.☆207Updated last week
- ☆107Updated last month
- ☆38Updated last month
- ☆147Updated 6 months ago
- Implement some method of LLM KV Cache Sparsity☆38Updated last year
- DLSlime: Flexible & Efficient Heterogeneous Transfer Toolkit☆67Updated this week
- ☆78Updated 5 months ago
- A sparse attention kernel supporting mix sparse patterns☆303Updated 7 months ago
- LLM Inference with Deep Learning Accelerator.☆50Updated 8 months ago
- Implement Flash Attention using Cute.☆95Updated 9 months ago
- Decoding Attention is specially optimized for MHA, MQA, GQA and MLA using CUDA core for the decoding stage of LLM inference.☆43Updated 3 months ago
- 16-fold memory access reduction with nearly no loss☆105Updated 6 months ago
- Official implementation of ICML 2024 paper "ExCP: Extreme LLM Checkpoint Compression via Weight-Momentum Joint Shrinking".☆48Updated last year
- qwen-nsa☆74Updated 5 months ago
- Compare different hardware platforms via the Roofline Model for LLM inference tasks.☆113Updated last year
- Accelerate LLM preference tuning via prefix sharing with a single line of code☆43Updated 2 months ago
- Efficient Mixture of Experts for LLM Paper List☆129Updated this week
- Code for paper: [ICLR2025 Oral] FlexPrefill: A Context-Aware Sparse Attention Mechanism for Efficient Long-Sequence Inference☆142Updated 4 months ago
- Quantized Attention on GPU☆44Updated 10 months ago
- ☆126Updated 3 months ago