jiaweizzhao / ZerO-initialization

Related projects ⓘ

Alternatives and complementary repositories for ZerO-initialization

• rasbt / cyclemoid-pytorch
  Cyclemoid implementation for PyTorch
  ☆87Updated 2 years ago
• lucidrains / compositional-attention-pytorch
  Implementation of "compositional attention" from MILA, a multi-head attention variant that is reframed as a two-step attention process wi…
  ☆50Updated 2 years ago
• lucidrains / autoregressive-linear-attention-cuda
  CUDA implementation of autoregressive linear attention, with all the latest research findings
  ☆43Updated last year
• lucidrains / gated-state-spaces-pytorch
  Implementation of Gated State Spaces, from the paper "Long Range Language Modeling via Gated State Spaces", in Pytorch
  ☆95Updated last year
• lucidrains / tableformer-pytorch
  Implementation of TableFormer, Robust Transformer Modeling for Table-Text Encoding, in Pytorch
  ☆36Updated 2 years ago
• lucidrains / kalman-filtering-attention
  Implementation of the Kalman Filtering Attention proposed in "Kalman Filtering Attention for User Behavior Modeling in CTR Prediction"
  ☆57Updated last year
• lucidrains / discrete-key-value-bottleneck-pytorch
  Implementation of Discrete Key / Value Bottleneck, in Pytorch
  ☆87Updated last year
• sayakpaul / MLP-Mixer-CIFAR10
  Implements MLP-Mixer (https://arxiv.org/abs/2105.01601) with the CIFAR-10 dataset.
  ☆54Updated 2 years ago
• lucidrains / panoptic-transformer
  Another attempt at a long-context / efficient transformer by me
  ☆37Updated 2 years ago
• lucidrains / Adan-pytorch
  Implementation of the Adan (ADAptive Nesterov momentum algorithm) Optimizer in Pytorch
  ☆247Updated 2 years ago
• ahennequ / pytorch-custom-mma
  ☆29Updated 2 years ago
• crowsonkb / torch-dist-utils
  Utilities for PyTorch distributed
  ☆23Updated last year
• pietrolesci / energizer
  An active learning library for Pytorch based on Lightning-Fabric.
  ☆79Updated 6 months ago
• lucidrains / logavgexp-torch
  Implementation of LogAvgExp for Pytorch
  ☆32Updated 2 years ago
• FrancescoSaverioZuppichini / Loading-huge-PyTorch-models-with-linear-memory-consumption
  Little article showing how to load pytorch's models with linear memory consumption
  ☆34Updated 2 years ago
• andravin / spio
  Efficient CUDA kernels for training convolutional neural networks with PyTorch.
  ☆35Updated this week
• HomebrewNLP / Olmax
  HomebrewNLP in JAX flavour for maintable TPU-Training
  ☆46Updated 10 months ago
• HomebrewNLP / HomebrewNLP
  A case study of efficient training of large language models using commodity hardware.
  ☆68Updated 2 years ago
• ClashLuke / tpucare
  Automatically take good care of your preemptible TPUs
  ☆32Updated last year
• lucidrains / gateloop-transformer
  Implementation of GateLoop Transformer in Pytorch and Jax
  ☆86Updated 5 months ago
• ag1988 / dss
  Sequence Modeling with Structured State Spaces
  ☆60Updated 2 years ago
• sayakpaul / big_vision_experiments
  Contains my experiments with the `big_vision` repo to train ViTs on ImageNet-1k.
  ☆22Updated last year
• gregorbachmann / scaling_mlps
  ☆51Updated 5 months ago
• shikaiqiu / compute-better-spent
  ☆46Updated last month
• lucidrains / light-recurrent-unit-pytorch
  Implementation of a Light Recurrent Unit in Pytorch
  ☆46Updated last month
• peerdavid / layerwise-batch-entropy
  Layerwise Batch Entropy Regularization
  ☆22Updated 2 years ago
• lucidrains / Mega-pytorch
  Implementation of Mega, the Single-head Attention with Multi-headed EMA architecture that currently holds SOTA on Long Range Arena
  ☆203Updated last year
• Zasder3 / open_clip_juwels
  An open source implementation of CLIP.
  ☆32Updated 2 years ago
• lucidrains / nystrom-attention
  Implementation of Nyström Self-attention, from the paper Nyströmformer
  ☆122Updated 10 months ago