☆48Aug 6, 2024Updated last year
Alternatives and similar repositories for Awesome-LLM-Training-System
Users that are interested in Awesome-LLM-Training-System are comparing it to the libraries listed below
Sorting:
- Website for CSE 234, Winter 2025☆13Mar 24, 2025Updated 11 months ago
- Implementation from scratch in C of the Multi-head latent attention used in the Deepseek-v3 technical paper.☆18Jan 15, 2025Updated last year
- An auxiliary project analysis of the characteristics of KV in DiT Attention.☆33Nov 29, 2024Updated last year
- [Archived] For the latest updates and community contribution, please visit: https://github.com/Ascend/TransferQueue or https://gitcode.co…☆13Jan 16, 2026Updated last month
- DeepGEMM: clean and efficient FP8 GEMM kernels with fine-grained scaling☆21Feb 9, 2026Updated 3 weeks ago
- TiledLower is a Dataflow Analysis and Codegen Framework written in Rust.☆14Nov 23, 2024Updated last year
- Paper reading and discussion notes, covering AI frameworks, distributed systems, cluster management, etc.☆55Nov 11, 2025Updated 3 months ago
- BMInf demos.☆16Oct 14, 2021Updated 4 years ago
- Triton-based Symmetric Memory operators and examples☆86Jan 15, 2026Updated last month
- ☆57Updated this week
- AI model training on heterogeneous, geo-distributed resources☆38Nov 24, 2025Updated 3 months ago
- Chimera: bidirectional pipeline parallelism for efficiently training large-scale models.☆70Mar 20, 2025Updated 11 months ago
- Awesome paper for multi-modal llm with grounding ability☆19Oct 11, 2025Updated 4 months ago
- ☆20Jul 5, 2024Updated last year
- High-performance distributed data shuffling (all-to-all) library for MoE training and inference☆112Updated this week
- NVSHMEM‑Tutorial: Build a DeepEP‑like GPU Buffer☆163Feb 11, 2026Updated 2 weeks ago
- ☆52May 19, 2025Updated 9 months ago
- Beyond KV Caching: Shared Attention for Efficient LLMs☆20Jul 19, 2024Updated last year
- ☆87Jan 22, 2026Updated last month
- DeepXTrace is a lightweight tool for precisely diagnosing slow ranks in DeepEP-based environments.☆93Jan 16, 2026Updated last month
- Supplemental materials for The ASPLOS 2025 / EuroSys 2025 Contest on Intra-Operator Parallelism for Distributed Deep Learning☆25May 12, 2025Updated 9 months ago
- The official implementation for the intra-stage fusion technique introduced in https://arxiv.org/abs/2409.13221☆30Apr 22, 2025Updated 10 months ago
- Tilus is a tile-level kernel programming language with explicit control over shared memory and registers.☆446Feb 4, 2026Updated 3 weeks ago
- A deep researcher agent that can take any topic and generate a research report in markdown with sources intact, all done locally!☆35Jun 27, 2025Updated 8 months ago
- ☆24Jul 7, 2024Updated last year
- Nex Venus Communication Library☆72Nov 17, 2025Updated 3 months ago
- Dynamic resources changes for multi-dimensional parallelism training☆30Aug 22, 2025Updated 6 months ago
- Curated collection of papers in MoE model inference☆343Oct 20, 2025Updated 4 months ago
- ☆131Nov 11, 2024Updated last year
- CUDA 8-bit Tensor Core Matrix Multiplication based on m16n16k16 WMMA API☆35Sep 15, 2023Updated 2 years ago
- Tiny-DeepSpeed, a minimalistic re-implementation of the DeepSpeed library☆50Aug 20, 2025Updated 6 months ago
- Asynchronous pipeline parallel optimization☆19Feb 2, 2026Updated last month
- Medusa: Accelerating Serverless LLM Inference with Materialization [ASPLOS'25]☆41May 13, 2025Updated 9 months ago
- ☆95Apr 2, 2025Updated 11 months ago
- NVIDIA NVSHMEM is a parallel programming interface for NVIDIA GPUs based on OpenSHMEM. NVSHMEM can significantly reduce multi-process com…☆469Updated this week
- Allow torch tensor memory to be released and resumed later☆220Feb 9, 2026Updated 3 weeks ago
- An efficient implementation of the NSA (Native Sparse Attention) kernel☆129Jun 24, 2025Updated 8 months ago
- ☆51Apr 30, 2025Updated 10 months ago
- MiniMax-Provider-Verifier offers a rigorous, vendor-agnostic way to verify whether third-party deployments of the Minimax M2 model are co…☆29Feb 18, 2026Updated last week