saareliad / FTPipe
FTPipe and related pipeline model parallelism research.
☆41Updated last year
Alternatives and similar repositories for FTPipe:
Users that are interested in FTPipe are comparing it to the libraries listed below
- ☆72Updated 3 years ago
- An Efficient Pipelined Data Parallel Approach for Training Large Model☆73Updated 4 years ago
- ☆35Updated 4 years ago
- ☆16Updated 2 years ago
- Research and development for optimizing transformers☆125Updated 4 years ago
- ☆43Updated last year
- ☆77Updated 2 years ago
- (NeurIPS 2022) Automatically finding good model-parallel strategies, especially for complex models and clusters.☆38Updated 2 years ago
- ☆79Updated 4 months ago
- Machine Learning System☆14Updated 4 years ago
- ☆26Updated 3 years ago
- An experimental parallel training platform☆54Updated last year
- Fairring (FAIR + Herring) is a plug-in for PyTorch that provides a process group for distributed training that outperforms NCCL at large …☆65Updated 3 years ago
- System for automated integration of deep learning backends.☆48Updated 2 years ago
- Chimera: bidirectional pipeline parallelism for efficiently training large-scale models.☆61Updated 2 weeks ago
- Analyze network performance in distributed training☆18Updated 4 years ago
- nnScaler: Compiling DNN models for Parallel Training☆103Updated last month
- Synthesizer for optimal collective communication algorithms☆105Updated 11 months ago
- PipeTransformer: Automated Elastic Pipelining for Distributed Training of Large-scale Models. ICML 2021☆56Updated 3 years ago
- ☆24Updated last year
- Boost hardware utilization for ML training workloads via Inter-model Horizontal Fusion☆32Updated 10 months ago
- Ultra | Ultimate | Unified CCL☆54Updated last month
- PipeSwitch: Fast Pipelined Context Switching for Deep Learning Applications☆126Updated 2 years ago
- Cavs: An Efficient Runtime System for Dynamic Neural Networks☆14Updated 4 years ago
- ☆9Updated last year
- ☆21Updated 2 years ago
- Fine-grained GPU sharing primitives☆141Updated 5 years ago
- AlpaServe: Statistical Multiplexing with Model Parallelism for Deep Learning Serving (OSDI 23)☆82Updated last year
- ☆53Updated 4 years ago
- ☆47Updated 2 years ago