Alternatives and similar repositories for IJCAI2023-OptimalShardedDataParallel

Users that are interested in IJCAI2023-OptimalShardedDataParallel are comparing it to the libraries listed below

• TonyTangYu / pytorch

  View on GitHub
  DELTA-pytorch：DELTA: Dynamically Optimizing GPU Memory beyond Tensor Recomputation
  ☆12Apr 16, 2024Updated last year
• helmholtz-analytics / mpi4torch

  View on GitHub
  An MPI wrapper for the pytorch tensor library that is automatically differentiable
  ☆10Mar 27, 2023Updated 2 years ago
• JF-D / Proteus

  View on GitHub
  ☆24Jul 7, 2024Updated last year
• awslabs / slapo

  View on GitHub
  A schedule language for large model training
  ☆152Aug 21, 2025Updated 5 months ago
• parasailteam / coconet

  View on GitHub
  ☆84Dec 2, 2022Updated 3 years ago
• xbfu / PyTorch-ParameterServer

  View on GitHub
  An implementation of parameter server framework in PyTorch RPC.
  ☆12Nov 12, 2021Updated 4 years ago
• aws-neuron / aws-neuron-reference-for-megatron-lm

  View on GitHub
  ☆14Aug 29, 2023Updated 2 years ago
• mw9385 / Hierarchical_Coverage_Path_Planning

  View on GitHub
  ☆12Apr 22, 2022Updated 3 years ago
• hku-systems / naspipe

  View on GitHub
  ☆14Jan 12, 2022Updated 4 years ago
• zhuohan123 / terapipe

  View on GitHub
  ☆77May 4, 2021Updated 4 years ago
• awslabs / ratex

  View on GitHub
  ☆23Aug 21, 2025Updated 5 months ago
• sail-sg / zero-bubble-pipeline-parallelism

  View on GitHub
  Zero Bubble Pipeline Parallelism
  ☆449May 7, 2025Updated 9 months ago
• ltricot / zerosum

  View on GitHub
  CFR implementation of a poker bot.
  ☆12Feb 17, 2023Updated 2 years ago
• HPDL-Group / Merak

  View on GitHub
  ☆82Updated this week
• MayDomine / Burst-Attention

  View on GitHub
  Distributed IO-aware Attention algorithm
  ☆24Sep 24, 2025Updated 4 months ago
• msr-fiddle / CheckFreq

  View on GitHub
  ☆56Jan 25, 2021Updated 5 years ago
• tlc-pack / libflash_attn

  View on GitHub
  Standalone Flash Attention v2 kernel without libtorch dependency
  ☆114Sep 10, 2024Updated last year
• natetsang / open-rl

  View on GitHub
  Implementations of a large collection of reinforcement learning algorithms.
  ☆28Nov 30, 2023Updated 2 years ago
• Yinghan-Li / YHs_Sample

  View on GitHub
  Yinghan's Code Sample
  ☆365Jul 25, 2022Updated 3 years ago
• kwai / Megatron-Kwai

  View on GitHub
  LLM training technologies developed by kwai
  ☆70Jan 21, 2026Updated 3 weeks ago
• pmodels / bolt

  View on GitHub
  Official BOLT Repository
  ☆31Aug 16, 2024Updated last year
• raharth / PyMatch

  View on GitHub
  A collection of different PyTorch wrappers for training neural networks and reinforcement algorithms
  ☆13Dec 15, 2022Updated 3 years ago
• thunlp / Seq1F1B

  View on GitHub
  Sequence-level 1F1B schedule for LLMs.
  ☆38Aug 26, 2025Updated 5 months ago
• rehbergT / dgemm

  View on GitHub
  Tutorial: Writing R and Python Packages with Multithreaded C++ Code using BLAS, AVX2/AVX512, OpenMP, C++11 Threads and Cuda GPU accelerat…
  ☆13Nov 27, 2022Updated 3 years ago
• Hsword / Hetu

  View on GitHub
  A high-performance distributed deep learning system targeting large-scale and automated distributed training. If you have any interests, …
  ☆124Dec 18, 2023Updated 2 years ago
• fzyzcjy / torch_memory_saver

  View on GitHub
  Allow torch tensor memory to be released and resumed later
  ☆217Updated this week
• hpcgarage / cuASR

  View on GitHub
  cuASR: CUDA Algebra for Semirings
  ☆44Aug 22, 2022Updated 3 years ago
• bdhirsh / pytorch_open_registration_example

  View on GitHub
  Example of using pytorch's open device registration API
  ☆31Oct 14, 2022Updated 3 years ago
• LINs-lab / DeFT

  View on GitHub
  [ICLR 2025] DeFT: Decoding with Flash Tree-attention for Efficient Tree-structured LLM Inference
  ☆49Jun 17, 2025Updated 7 months ago
• zhisbug / ray-scalable-ml-design

  View on GitHub
  Some microbenchmarks and design docs before commencement
  ☆12Feb 1, 2021Updated 5 years ago
• fpgasystems / hacc

  View on GitHub
  ETHZ Heterogeneous Accelerated Compute Cluster.
  ☆38Oct 7, 2025Updated 4 months ago
• heheda12345 / MagPy

  View on GitHub
  ☆41Jun 5, 2024Updated last year
• microsoft / NPKit

  View on GitHub
  NCCL Profiling Kit
  ☆152Jul 1, 2024Updated last year
• MarcoMeter / neroRL

  View on GitHub
  Deep Reinforcement Learning Framework done with PyTorch
  ☆43Mar 12, 2025Updated 11 months ago
• littleSunlxy / FedPU-torch

  View on GitHub
  Official PyTorch Implementation of Federated Learning with Positive and Unlabeled Data
  ☆10Aug 12, 2022Updated 3 years ago
• MLblog / jads_kaggle

  View on GitHub
  Contains our group's work in various kaggle competitions
  ☆11Nov 8, 2019Updated 6 years ago
• Picsart-AI-Research / Social-Reward

  View on GitHub
  [ICLR 2024 Spotlight] Social Reward: Evaluating and Enhancing Generative AI through Million-User Feedback from an Online Creative Communi…
  ☆11Mar 29, 2024Updated last year
• rcrowder / AdaptiveResonanceTheory

  View on GitHub
  ADAPTIVE RESONANCE THEORY. Gail A. Carpenter and Stephen Grossberg
  ☆10Feb 10, 2015Updated 11 years ago
• ACIL-Group / NuART-Py

  View on GitHub
  NuART-Py: Python Library of Adaptive Resonance Theory Neural Network
  ☆10Jan 26, 2020Updated 6 years ago