Alternatives and similar repositories for gradient-checkpointing

Users that are interested in gradient-checkpointing are comparing it to the libraries listed below

• tensorflow / mesh
  Mesh TensorFlow: Model Parallelism Made Easier
  ☆1,608Updated last year
• kakaobrain / torchgpipe
  A GPipe implementation in PyTorch
  ☆843Updated 10 months ago
• Stonesjtu / pytorch_memlab
  Profiling and inspecting memory in pytorch
  ☆1,061Updated 10 months ago
• facebookresearch / fairscale
  PyTorch extensions for high performance and large scale training.
  ☆3,331Updated last month
• openai / blocksparse
  Efficient GPU kernels for block-sparse matrix multiplication and convolution
  ☆1,041Updated 2 years ago
• openai / sparse_attention
  Examples of using sparse attention, as in "Generating Long Sequences with Sparse Transformers"
  ☆1,577Updated 4 years ago
• pytorch / tnt
  A lightweight library for PyTorch training tools and utilities
  ☆1,699Updated last week
• bckenstler / CLR
  ☆1,209Updated 5 years ago
• prigoyal / pytorch_memonger
  Experimental ground for optimizing memory of pytorch models
  ☆366Updated 7 years ago
• jettify / pytorch-optimizer
  torch-optimizer -- collection of optimizers for Pytorch
  ☆3,115Updated last year
• idiap / fast-transformers
  Pytorch library for fast transformer implementations
  ☆1,714Updated 2 years ago
• NVIDIA / DALI
  A GPU-accelerated library containing highly optimized building blocks and an execution engine for data processing to accelerate deep lear…
  ☆5,434Updated this week
• pytorch / ignite
  High-level library to help with training and evaluating neural networks in PyTorch flexibly and transparently.
  ☆4,668Updated this week
• LiyuanLucasLiu / RAdam
  On the Variance of the Adaptive Learning Rate and Beyond
  ☆2,549Updated 3 years ago
• waleedka / hiddenlayer
  Neural network graphs and training metrics for PyTorch, Tensorflow, and Keras.
  ☆1,832Updated last year
• pytorch / elastic
  PyTorch elastic training
  ☆728Updated 3 years ago
• Santosh-Gupta / SpeedTorch
  Library for faster pinned CPU <-> GPU transfer in Pytorch
  ☆685Updated 5 years ago
• webdataset / webdataset
  A high-performance Python-based I/O system for large (and small) deep learning problems, with strong support for PyTorch.
  ☆2,639Updated 3 weeks ago
• NVIDIA / apex
  A PyTorch Extension: Tools for easy mixed precision and distributed training in Pytorch
  ☆8,686Updated this week
• microsoft / fastformers
  FastFormers - highly efficient transformer models for NLU
  ☆705Updated 3 months ago
• davidcpage / cifar10-fast
  ☆536Updated 3 years ago
• asappresearch / sru
  Training RNNs as Fast as CNNs (https://arxiv.org/abs/1709.02755)
  ☆2,098Updated 3 years ago
• mlcommons / training
  Reference implementations of MLPerf™ training benchmarks
  ☆1,683Updated last month
• pytorch / xla
  Enabling PyTorch on XLA Devices (e.g. Google TPU)
  ☆2,623Updated this week
• Luolc / AdaBound
  An optimizer that trains as fast as Adam and as good as SGD.
  ☆2,916Updated last year
• google / gin-config
  Gin provides a lightweight configuration framework for Python
  ☆2,107Updated last month
• lucidrains / reformer-pytorch
  Reformer, the efficient Transformer, in Pytorch
  ☆2,170Updated 2 years ago
• harvardnlp / pytorch-struct
  Fast, general, and tested differentiable structured prediction in PyTorch
  ☆1,113Updated 3 years ago
• pytorchbearer / torchbearer
  torchbearer: A model fitting library for PyTorch
  ☆640Updated last year
• ilkarman / DeepLearningFrameworks
  Demo of running NNs across different frameworks
  ☆1,652Updated 2 years ago