saareliad / FTPipe
FTPipe and related pipeline model parallelism research.
☆41Updated last year
Related projects ⓘ
Alternatives and complementary repositories for FTPipe
- ☆65Updated 3 years ago
- An Efficient Pipelined Data Parallel Approach for Training Large Model☆70Updated 3 years ago
- Research and development for optimizing transformers☆124Updated 3 years ago
- (NeurIPS 2022) Automatically finding good model-parallel strategies, especially for complex models and clusters.☆34Updated 2 years ago
- ☆72Updated last year
- nnScaler: Compiling DNN models for Parallel Training☆62Updated 2 weeks ago
- ☆16Updated last year
- A schedule language for large model training☆141Updated 4 months ago
- An experimental parallel training platform☆52Updated 7 months ago
- ☆26Updated 3 years ago
- PET: Optimizing Tensor Programs with Partially Equivalent Transformations and Automated Corrections☆114Updated 2 years ago
- ☆44Updated last year
- Chimera: Efficiently Training Large-Scale Neural Networks with Bidirectional Pipelines.☆44Updated 11 months ago
- PipeSwitch: Fast Pipelined Context Switching for Deep Learning Applications☆124Updated 2 years ago
- ☆35Updated 3 years ago
- ☆74Updated 2 weeks ago
- ☆47Updated last year
- Synthesizer for optimal collective communication algorithms☆98Updated 7 months ago
- ☆31Updated last year
- Set of datasets for the deep learning recommendation model (DLRM).☆40Updated last year
- Fairring (FAIR + Herring) is a plug-in for PyTorch that provides a process group for distributed training that outperforms NCCL at large …☆63Updated 2 years ago
- PipeTransformer: Automated Elastic Pipelining for Distributed Training of Large-scale Models. ICML 2021☆54Updated 3 years ago
- 🔮 Execution time predictions for deep neural network training iterations across different GPUs.☆54Updated last year
- Boost hardware utilization for ML training workloads via Inter-model Horizontal Fusion☆32Updated 5 months ago
- ☆70Updated 2 years ago
- AlpaServe: Statistical Multiplexing with Model Parallelism for Deep Learning Serving (OSDI 23)☆79Updated last year
- ☆8Updated last year
- System for automated integration of deep learning backends.☆48Updated 2 years ago
- Simple Distributed Deep Learning on TensorFlow☆134Updated 2 years ago
- PArametrized Recommendation and Ai Model benchmark is a repository for development of numerous uBenchmarks as well as end to end nets for…☆122Updated this week