kaiyuyue / torchshardLinks
Slicing a PyTorch Tensor Into Parallel Shards
☆299Updated 3 years ago
Alternatives and similar repositories for torchshard
Users that are interested in torchshard are comparing it to the libraries listed below
Sorting:
- Accelerate training by storing parameters in one contiguous chunk of memory.☆291Updated 4 years ago
- Demystify RAM Usage in Multi-Process Data Loaders☆194Updated 2 years ago
- ActNN: Reducing Training Memory Footprint via 2-Bit Activation Compressed Training☆200Updated 2 years ago
- Implementation of a Transformer, but completely in Triton☆265Updated 3 years ago
- Experimental ground for optimizing memory of pytorch models☆365Updated 7 years ago
- PyTorch layer-by-layer model profiler☆607Updated 4 years ago
- Implementation of https://arxiv.org/abs/1904.00962☆374Updated 4 years ago
- A general and accurate MACs / FLOPs profiler for PyTorch models☆613Updated last year
- A GPU performance profiling tool for PyTorch models☆503Updated 3 years ago
- Example code showing how to use Nvidia DALI in pytorch, with fallback to torchvision. Contains a few differences to the official Nvidia …☆197Updated 5 years ago
- A LARS implementation in PyTorch☆345Updated 5 years ago
- A GPipe implementation in PyTorch☆842Updated 10 months ago
- ☆261Updated 5 years ago
- MONeT framework for reducing memory consumption of DNN training☆173Updated 4 years ago
- [Prototype] Tools for the concurrent manipulation of variably sized Tensors.☆251Updated 2 years ago
- Torch Distributed Experimental☆117Updated 9 months ago
- Library for 8-bit optimizers and quantization routines.☆716Updated 2 years ago
- Fast Block Sparse Matrices for Pytorch☆545Updated 4 years ago
- [ACL'20] HAT: Hardware-Aware Transformers for Efficient Natural Language Processing☆334Updated 10 months ago
- Block-sparse primitives for PyTorch☆155Updated 4 years ago
- Tutorial for building a custom CUDA function for Pytorch☆515Updated 6 years ago
- TVM integration into PyTorch☆452Updated 5 years ago
- torch::deploy (multipy for non-torch uses) is a system that lets you get around the GIL problem by running multiple Python interpreters i…☆178Updated 5 months ago
- Official PyTorch Implementation of Long-Short Transformer (NeurIPS 2021).☆225Updated 3 years ago
- ☆165Updated 6 years ago
- ☆109Updated 4 years ago
- Memory Efficient Attention (O(sqrt(n)) for Jax and PyTorch☆183Updated 2 years ago
- Official code for "Writing Distributed Applications with PyTorch", PyTorch Tutorial☆262Updated 2 years ago
- Estimate/count FLOPS for a given neural network using pytorch☆304Updated 3 years ago
- PyTorch implementation of L2L execution algorithm☆107Updated 2 years ago