tensorflow / meshLinks
Mesh TensorFlow: Model Parallelism Made Easier
☆1,615Updated last year
Alternatives and similar repositories for mesh
Users that are interested in mesh are comparing it to the libraries listed below
Sorting:
- PyTorch extensions for high performance and large scale training.☆3,361Updated 3 months ago
- PyTorch elastic training☆729Updated 3 years ago
- Efficient GPU kernels for block-sparse matrix multiplication and convolution☆1,045Updated 2 years ago
- Make huge neural nets fit in memory☆2,809Updated 5 years ago
- Reference implementations of MLPerf™ training benchmarks☆1,702Updated this week
- Examples of using sparse attention, as in "Generating Long Sequences with Sparse Transformers"☆1,584Updated 5 years ago
- Collective communications library with various primitives for multi-machine training.☆1,342Updated 3 weeks ago
- Enabling PyTorch on XLA Devices (e.g. Google TPU)☆2,653Updated this week
- Central place for the engineering/scaling WG: documentation, SLURM scripts and logs, compute environment and data.☆1,006Updated last year
- A GPipe implementation in PyTorch☆849Updated last year
- a fast and user-friendly runtime for transformer inference (Bert, Albert, GPT2, Decoders, etc) on CPU and GPU.☆1,530Updated last month
- Long Range Arena for Benchmarking Efficient Transformers☆762Updated last year
- A performant and modular runtime for TensorFlow☆758Updated 2 weeks ago
- Fast Block Sparse Matrices for Pytorch☆549Updated 4 years ago
- Lingvo☆2,852Updated 2 months ago
- Dataset, streaming, and file system extensions maintained by TensorFlow SIG-IO☆733Updated this week
- Providing reproducibility in deep learning frameworks☆428Updated last year
- FB (Facebook) + GEMM (General Matrix-Matrix Multiplication) - https://code.fb.com/ml-applications/fbgemm/☆1,426Updated this week
- Parallelformers: An Efficient Model Parallelization Toolkit for Deployment☆791Updated 2 years ago
- PatrickStar enables Larger, Faster, Greener Pretrained Models for NLP and democratizes AI for everyone.☆764Updated 2 years ago
- Making text a first-class citizen in TensorFlow.☆1,270Updated this week
- Bagua Speeds up PyTorch☆884Updated last year
- TorchBench is a collection of open source benchmarks used to evaluate PyTorch performance.☆976Updated this week
- ☆2,863Updated 2 months ago
- Task-based datasets, preprocessing, and evaluation for sequence models.☆585Updated last week
- Ongoing research training transformer language models at scale, including: BERT & GPT-2☆1,411Updated last year
- Training and serving large-scale neural networks with auto parallelization.☆3,148Updated last year
- Kernl lets you run PyTorch transformer models several times faster on GPU with a single line of code, and is designed to be easily hackab…☆1,581Updated last year
- Useful extra functionality for TensorFlow 2.x maintained by SIG-addons☆1,701Updated 4 months ago
- maximal update parametrization (µP)☆1,584Updated last year