[IJCAI2023] An automated parallel training system that combines the advantages from both data and model parallelism. If you have any interests, please visit/star/fork https://github.com/Youhe-Jiang/OptimalShardedDataParallel
☆52May 31, 2023Updated 2 years ago
Alternatives and similar repositories for IJCAI2023-OptimalShardedDataParallel
Users that are interested in IJCAI2023-OptimalShardedDataParallel are comparing it to the libraries listed below
Sorting:
- DELTA-pytorch：DELTA: Dynamically Optimizing GPU Memory beyond Tensor Recomputation☆12Apr 16, 2024Updated last year
- An MPI wrapper for the pytorch tensor library that is automatically differentiable☆10Mar 27, 2023Updated 2 years ago
- Bagua tutorials.☆13Sep 4, 2022Updated 3 years ago
- ☆24Jul 7, 2024Updated last year
- A schedule language for large model training☆152Aug 21, 2025Updated 6 months ago
- ☆84Dec 2, 2022Updated 3 years ago
- ☆14Aug 29, 2023Updated 2 years ago
- ☆14Jan 12, 2022Updated 4 years ago
- ☆17Feb 19, 2024Updated 2 years ago
- ☆78May 4, 2021Updated 4 years ago
- ☆23Aug 21, 2025Updated 6 months ago
- High performance NCCL plugin for Bagua.☆15Sep 15, 2021Updated 4 years ago
- ☆19Oct 31, 2022Updated 3 years ago
- CFR implementation of a poker bot.☆12Feb 17, 2023Updated 3 years ago
- ☆82Feb 11, 2026Updated 3 weeks ago
- Distributed IO-aware Attention algorithm☆24Sep 24, 2025Updated 5 months ago
- Galvatron is an automatic distributed training system designed for Transformer models, including Large Language Models (LLMs). If you hav…☆23Oct 22, 2025Updated 4 months ago
- ☆56Jan 25, 2021Updated 5 years ago
- Deep Learning Project☆23Jan 18, 2020Updated 6 years ago
- Standalone Flash Attention v2 kernel without libtorch dependency☆114Sep 10, 2024Updated last year
- Implementations of a large collection of reinforcement learning algorithms.☆28Nov 30, 2023Updated 2 years ago
- Yinghan's Code Sample☆365Jul 25, 2022Updated 3 years ago
- ☆74Oct 31, 2024Updated last year
- Tutorial: Writing R and Python Packages with Multithreaded C++ Code using BLAS, AVX2/AVX512, OpenMP, C++11 Threads and Cuda GPU accelerat…☆13Nov 27, 2022Updated 3 years ago
- A high-performance distributed deep learning system targeting large-scale and automated distributed training. If you have any interests, …☆124Dec 18, 2023Updated 2 years ago
- cuASR: CUDA Algebra for Semirings☆45Aug 22, 2022Updated 3 years ago
- (NeurIPS 2022) Automatically finding good model-parallel strategies, especially for complex models and clusters.☆44Nov 4, 2022Updated 3 years ago
- Example of using pytorch's open device registration API☆31Oct 14, 2022Updated 3 years ago
- ETHZ Heterogeneous Accelerated Compute Cluster.☆38Oct 7, 2025Updated 5 months ago
- Some microbenchmarks and design docs before commencement☆12Feb 1, 2021Updated 5 years ago
- extensible collectives library in triton☆96Mar 31, 2025Updated 11 months ago
- ☆41Jun 5, 2024Updated last year
- NCCL Profiling Kit☆152Jul 1, 2024Updated last year
- Deep Reinforcement Learning Framework done with PyTorch☆43Mar 12, 2025Updated 11 months ago
- Off-Platform Application example for ONOS Intent Monitor and Reroute service☆11Jul 23, 2019Updated 6 years ago
- A Texas Holdem poker framework written in C++ 20.☆11Apr 23, 2023Updated 2 years ago
- Official PyTorch Implementation of Federated Learning with Positive and Unlabeled Data☆10Aug 12, 2022Updated 3 years ago
- 🌿快速生成文件夹目录结构，支持定义目录层级，支持生成到 markdown 文件。☆13Oct 19, 2022Updated 3 years ago
- Source code for journal paper "Multiagent Reinforcement Learning With Sparse Interactions by Negotiation and Knowledge Transfer"☆13Dec 26, 2017Updated 8 years ago