spcl / sten
Sparsity support for PyTorch
☆31Updated this week
Related projects ⓘ
Alternatives and complementary repositories for sten
- Fast Hadamard transform in CUDA, with a PyTorch interface☆111Updated 6 months ago
- Memory Optimizations for Deep Learning (ICML 2023)☆60Updated 8 months ago
- Distributed K-FAC Preconditioner for PyTorch☆80Updated this week
- (NeurIPS 2022) Automatically finding good model-parallel strategies, especially for complex models and clusters.☆34Updated 2 years ago
- extensible collectives library in triton☆72Updated 2 months ago
- ☆23Updated 10 months ago
- ☆66Updated 3 years ago
- ☆36Updated last year
- ☆88Updated 2 months ago
- ☆15Updated 2 years ago
- ☆20Updated last week
- ☆15Updated 5 years ago
- ☆46Updated 5 months ago
- ☆11Updated 2 years ago
- ☆45Updated 2 weeks ago
- Research and development for optimizing transformers☆125Updated 3 years ago
- Python package for rematerialization-aware gradient checkpointing☆23Updated last year
- A parallel framework for training deep neural networks☆45Updated 2 weeks ago
- ☆33Updated last year
- Triton-based implementation of Sparse Mixture of Experts.☆185Updated last month
- Cavs: An Efficient Runtime System for Dynamic Neural Networks☆13Updated 4 years ago
- ☆23Updated 2 months ago
- Experiment of using Tangent to autodiff triton☆72Updated 10 months ago
- NAACL '24 (Best Demo Paper RunnerUp) / MlSys @ NeurIPS '23 - RedCoast: A Lightweight Tool to Automate Distributed Training and Inference☆61Updated last month
- Chimera: Efficiently Training Large-Scale Neural Networks with Bidirectional Pipelines.☆46Updated 11 months ago
- ☆132Updated 4 months ago
- PyTorch bindings for CUTLASS grouped GEMM.☆53Updated 3 weeks ago
- ☆22Updated 11 months ago
- PipeTransformer: Automated Elastic Pipelining for Distributed Training of Large-scale Models. ICML 2021☆55Updated 3 years ago
- ☆55Updated 6 months ago