Alternatives and similar repositories for online-normalization:

Users that are interested in online-normalization are comparing it to the libraries listed below

• Separius / CudaRelativeAttention
  custom cuda kernel for {2, 3}d relative attention with pytorch wrapper
  ☆43Updated 4 years ago
• adityaiitb / pyprof2
  PyProf2: PyTorch Profiling tool
  ☆82Updated 4 years ago
• angelajiang / SelectiveBackprop
  SelectiveBackprop accelerates training by dynamically prioritizing useful examples with high loss
  ☆32Updated 4 years ago
• davda54 / ada-hessian
  Easy-to-use AdaHessian optimizer (PyTorch)
  ☆77Updated 4 years ago
• jxbz / madam
  👩 Pytorch and Jax code for the Madam optimiser.
  ☆51Updated 3 years ago
• renqianluo / GBDT-NAS
  GBDT-NAS
  ☆28Updated 3 years ago
• lucidrains / mlp-gpt-jax
  A GPT, made only of MLPs, in Jax
  ☆57Updated 3 years ago
• HazyResearch / structured-nets
  Structured matrices for compressing neural networks
  ☆66Updated last year
• izmailovpavel / torch_swa_examples
  ☆47Updated 4 years ago
• fabio-deep / ReZero-ResNet
  Unofficial pytorch implementation of ReZero in ResNet
  ☆23Updated 4 years ago
• YuchenJin / autolrs
  Automatic learning-rate scheduler
  ☆44Updated 3 years ago
• NVlabs / webloader
  Efficient DataLoader for PyTorch and Keras for loading datasets from web servers and object stores.
  ☆29Updated 5 years ago
• speechmatics / hqa
  Code to accompany the paper "Hierarchical Quantized Autoencoders"
  ☆37Updated last year
• f-dangel / sirfshampoo
  [ICML 2024] SIRFShampoo: Structured inverse- and root-free Shampoo in PyTorch (https://arxiv.org/abs/2402.03496)
  ☆14Updated 2 months ago
• jongwook / tfrecord_lite
  Make TFRecord Usable Again
  ☆87Updated last year
• minhtannguyen / SRSGD
  Code base for SRSGD.
  ☆28Updated 4 years ago
• tk-rusch / LEM
  Official code for Long Expressive Memory (ICLR 2022, Spotlight)
  ☆69Updated 2 years ago
• peerdavid / layerwise-batch-entropy
  Layerwise Batch Entropy Regularization
  ☆22Updated 2 years ago
• moskomule / shampoo.pytorch
  An implementation of shampoo
  ☆74Updated 6 years ago
• mit-han-lab / neurips-micronet
  [JMLR'20] NeurIPS 2019 MicroNet Challenge Efficient Language Modeling, Champion
  ☆40Updated 3 years ago
• L0SG / NanoFlow
  PyTorch implementation of the paper "NanoFlow: Scalable Normalizing Flows with Sublinear Parameter Complexity." (NeurIPS 2020)
  ☆64Updated 4 years ago
• noahgolmant / pytorch-lars
  "Layer-wise Adaptive Rate Scaling" in PyTorch
  ☆86Updated 4 years ago
• tbachlechner / ReZero-examples
  PyTorch Examples repo for "ReZero is All You Need: Fast Convergence at Large Depth"
  ☆62Updated 6 months ago
• hongyi-zhang / Fixup
  A Re-implementation of Fixed-update Initialization
  ☆152Updated 5 years ago
• ppwwyyxx / FRN-on-common-ImageNet-baseline
  Filter Response Normalization tested on better ImageNet baselines.
  ☆35Updated 4 years ago
• stevenygd / SWALP
  Code for paper "SWALP: Stochastic Weight Averaging forLow-Precision Training".
  ☆62Updated 5 years ago
• ag14774 / diffdist
  ☆62Updated 4 years ago
• YiwenShaoStephen / NGD-SGD
  A Pytorch Implementation of Natural Gradient Descent
  ☆44Updated 5 years ago
• shivram1987 / diffGrad
  diffGrad: An Optimization Method for Convolutional Neural Networks
  ☆55Updated 2 years ago
• teddykoker / performer
  Simply Numpy implementation of the FAVOR+ attention mechanism, https://teddykoker.com/2020/11/performers/
  ☆37Updated 4 years ago