xuqifan897 / Optimus
☆26Updated 3 years ago
Alternatives and similar repositories for Optimus:
Users that are interested in Optimus are comparing it to the libraries listed below
- ☆72Updated 3 years ago
- FTPipe and related pipeline model parallelism research.☆41Updated last year
- (NeurIPS 2022) Automatically finding good model-parallel strategies, especially for complex models and clusters.☆38Updated 2 years ago
- A Python library transfers PyTorch tensors between CPU and NVMe☆111Updated 3 months ago
- Research and development for optimizing transformers☆125Updated 4 years ago
- ☆81Updated 2 years ago
- ☆79Updated 3 months ago
- ☆44Updated last year
- PyTorch bindings for CUTLASS grouped GEMM.☆108Updated 2 months ago
- ☆75Updated 2 years ago
- Chimera: bidirectional pipeline parallelism for efficiently training large-scale models.☆60Updated this week
- ☆116Updated last year
- PyTorch bindings for CUTLASS grouped GEMM.☆74Updated 4 months ago
- ☆62Updated 3 weeks ago
- Performance benchmarking with ColossalAI☆39Updated 2 years ago
- An Efficient Pipelined Data Parallel Approach for Training Large Model☆74Updated 4 years ago
- ☆88Updated 6 months ago
- [USENIX ATC '24] Accelerating the Training of Large Language Models using Efficient Activation Rematerialization and Optimal Hybrid Paral…☆51Updated 7 months ago
- A baseline repository of Auto-Parallelism in Training Neural Networks☆143Updated 2 years ago
- Official repository for LightSeq: Sequence Level Parallelism for Distributed Training of Long Context Transformers☆207Updated 7 months ago
- ☆101Updated 6 months ago
- ☆55Updated 2 months ago
- ☆16Updated 2 years ago
- Odysseus: Playground of LLM Sequence Parallelism☆66Updated 9 months ago
- This repository contains the results and code for the MLPerf™ Training v1.0 benchmark.☆38Updated last year
- Flash-LLM: Enabling Cost-Effective and Highly-Efficient Large Generative Model Inference with Unstructured Sparsity☆202Updated last year
- ☆73Updated 4 months ago
- Python package for rematerialization-aware gradient checkpointing☆24Updated last year
- PipeTransformer: Automated Elastic Pipelining for Distributed Training of Large-scale Models. ICML 2021☆56Updated 3 years ago
- nnScaler: Compiling DNN models for Parallel Training☆103Updated last month