Alternatives and similar repositories for DeepSpeed

Users that are interested in DeepSpeed are comparing it to the libraries listed below

• fanshiqing / grouped_gemm
  PyTorch bindings for CUTLASS grouped GEMM.
  ☆184Updated last month
• flagos-ai / FlagAttention
  A collection of memory efficient attention operators implemented in the Triton language.
  ☆287Updated last year
• Oneflow-Inc / libai
  LiBai(李白): A Toolbox for Large-Scale Distributed Parallel Training
  ☆405Updated 6 months ago
• NVlabs / COAT
  [ICLR 2025] COAT: Compressing Optimizer States and Activation for Memory-Efficient FP8 Training
  ☆258Updated 5 months ago
• flagos-ai / FlagScale
  FlagScale is a large model toolkit based on open-sourced projects.
  ☆471Updated last week
• bytedance / ByteTransformer
  optimized BERT transformer inference on NVIDIA GPU. https://arxiv.org/abs/2210.03052
  ☆476Updated last year
• intelligent-machine-learning / atorch
  An industrial extension library of pytorch to accelerate large scale model training
  ☆58Updated 5 months ago
• OpenPPL / ppl.llm.serving
  ☆130Updated last year
• DeepLink-org / dlinfer
  ☆74Updated this week
• Ascend / pytorch
  Ascend PyTorch adapter (torch_npu). Mirror of https://gitee.com/ascend/pytorch
  ☆483Updated last week
• feifeibear / long-context-attention
  USP: Unified (a.k.a. Hybrid, 2D) Sequence Parallel Attention for Long Context Transformers Model Training and Inference
  ☆641Updated 3 weeks ago
• void-main / FasterTransformer
  Transformer related optimization, including BERT, GPT
  ☆59Updated 2 years ago
• OpenPPL / ppl.nn.llm
  ☆141Updated last year
• Ascend / AscendSpeed
  ☆79Updated 2 years ago
• sail-sg / zero-bubble-pipeline-parallelism
  Zero Bubble Pipeline Parallelism
  ☆449Updated 9 months ago
• volcengine / veGiantModel
  ☆219Updated 2 years ago
• megvii-research / Sparsebit
  A model compression and acceleration toolbox based on pytorch.
  ☆333Updated 2 years ago
• THUDM / FasterTransformer
  Transformer related optimization, including BERT, GPT
  ☆39Updated 2 years ago
• openmlsys / openmlsys-cuda
  Tutorials for writing high-performance GPU operators in AI frameworks.
  ☆136Updated 2 years ago
• Oneflow-Inc / OneFlow-Benchmark
  OneFlow models for benchmarking.
  ☆104Updated last year
• madsys-dev / deepseekv2-profile
  ☆155Updated 11 months ago
• InternLM / InternEvo
  InternEvo is an open-sourced lightweight training framework aims to support model pre-training without the need for extensive dependencie…
  ☆417Updated 5 months ago
• ModelTC / rank_dataset
  PyTorch Dataset Rank Dataset
  ☆43Updated 4 years ago
• BBuf / megatron-lm-parallel-group-playground
  ☆16Updated last year
• alibaba / EasyParallelLibrary
  Easy Parallel Library (EPL) is a general and efficient deep learning framework for distributed model training.
  ☆271Updated 2 years ago
• zhuzilin / ring-flash-attention
  Ring attention implementation with flash attention
  ☆979Updated 4 months ago
• modelscope / dash-infer
  DashInfer is a native LLM inference engine aiming to deliver industry-leading performance atop various hardware architectures, including …
  ☆274Updated 6 months ago
• Oneflow-Inc / models
  Models and examples built with OneFlow
  ☆101Updated last year
• CalvinXKY / mfu_calculation
  A simple calculation for LLM MFU.
  ☆66Updated 4 months ago
• AyakaGEMM / Hands-on-GEMM
  ☆145Updated last year