Alternatives and similar repositories for Megatron-Kwai:

Users that are interested in Megatron-Kwai are comparing it to the libraries listed below

• fanshiqing / grouped_gemm
  PyTorch bindings for CUTLASS grouped GEMM.
  ☆94Updated last month
• LoongServe / LoongServe
  ☆83Updated 3 months ago
• madsys-dev / deepseekv2-profile
  ☆101Updated 6 months ago
• zhuohan123 / terapipe
  ☆72Updated 3 years ago
• microsoft / nnscaler
  nnScaler: Compiling DNN models for Parallel Training
  ☆93Updated last week
• AlibabaPAI / FLASHNN
  ☆81Updated 5 months ago
• ConnollyLeon / awesome-Auto-Parallelism
  A baseline repository of Auto-Parallelism in Training Neural Networks
  ☆142Updated 2 years ago
• thunlp / Seq1F1B
  Sequence-level 1F1B schedule for LLMs.
  ☆16Updated last month
• HPDL-Group / Merak
  ☆79Updated 2 months ago
• OpenPPL / ppl.nn.llm
  ☆140Updated 9 months ago
• FlagOpen / FlagAttention
  A collection of memory efficient attention operators implemented in the Triton language.
  ☆237Updated 8 months ago
• bytedance / flux
  A fast communication-overlapping library for tensor parallelism on GPUs.
  ☆296Updated 3 months ago
• thu-pacman / SmartMoE-AE
  ATC23 AE
  ☆45Updated last year
• alibaba / easydist
  Automated Parallelization System and Infrastructure for Multiple Ecosystems
  ☆78Updated 3 months ago
• feifeibear / LLMRoofline
  Compare different hardware platforms via the Roofline Model for LLM inference tasks.
  ☆93Updated 11 months ago
• InternLM / Awesome-LLM-Training-System
  ☆24Updated 6 months ago
• LLMServe / SwiftTransformer
  High performance Transformer implementation in C++.
  ☆102Updated last month
• AlibabaPAI / torchacc
  PyTorch distributed training acceleration framework
  ☆39Updated last week
• MoE-Inf / awesome-moe-inference
  Curated collection of papers in MoE model inference
  ☆64Updated this week
• interestingLSY / swiftLLM
  A tiny yet powerful LLM inference system tailored for researching purpose. vLLM-equivalent performance with only 2k lines of code (2% of …
  ☆138Updated 7 months ago
• microsoft / vattention
  Dynamic Memory Management for Serving LLMs without PagedAttention
  ☆290Updated this week
• OpenPPL / ppl.llm.kernel.cuda
  ☆142Updated last month
• sail-sg / zero-bubble-pipeline-parallelism
  Zero Bubble Pipeline Parallelism
  ☆336Updated last week
• 66RING / tiny-flash-attention
  flash attention tutorial written in python, triton, cuda, cutlass
  ☆260Updated last month
• feifeibear / long-context-attention
  USP: Unified (a.k.a. Hybrid, 2D) Sequence Parallel Attention for Long Context Transformers Model Training and Inference
  ☆424Updated this week
• ParCIS / Chimera
  Chimera: Efficiently Training Large-Scale Neural Networks with Bidirectional Pipelines.
  ☆57Updated last year
• DicardoX / Research-Space
  This repository is established to store personal notes and annotated papers during daily research.
  ☆109Updated this week
• AyakaGEMM / Hands-on-GEMM
  ☆110Updated 11 months ago
• thu-pacman / FasterMoE
  ☆76Updated 2 years ago
• weishengying / cutlass_flash_atten_fp8
  使用 cutlass 仓库在 ada 架构上实现 fp8 的 flash attention
  ☆53Updated 6 months ago