Alternatives and similar repositories for Tri-RMSNorm

Users that are interested in Tri-RMSNorm are comparing it to the libraries listed below

• lucidrains / minGRU-pytorch
  Implementation of the proposed minGRU in Pytorch
  ☆319Updated 2 months ago
• proger / accelerated-scan
  Accelerated First Order Parallel Associative Scan
  ☆196Updated last month
• lucidrains / adam-atan2-pytorch
  Implementation of the proposed Adam-atan2 from Google Deepmind in Pytorch
  ☆135Updated 3 months ago
• kozistr / pytorch_optimizer
  optimizer & lr scheduler & loss function collections in PyTorch
  ☆387Updated this week
• HomebrewML / HeavyBall
  Efficient optimizers
  ☆281Updated last month
• Gunale0926 / Grams
  Grams: Gradient Descent with Adaptive Momentum Scaling (ICLR 2025 Workshop)
  ☆17Updated 11 months ago
• zyushun / Adam-mini
  Code for Adam-mini: Use Fewer Learning Rates To Gain More https://arxiv.org/abs/2406.16793
  ☆452Updated 8 months ago
• lucidrains / hyper-connections
  Attempt to make multiple residual streams from Bytedance's Hyper-Connections paper accessible to the public
  ☆172Updated last week
• taehong-moon / ee-diffusion
  Official Code Implementation for 'A Simple Early Exiting Framework for Accelerated Sampling in Diffusion Models'
  ☆20Updated last year
• zzd1992 / FlashWindowAttention
  Speedup the attention computation of Swin Transformer
  ☆31Updated 7 months ago
• juice500ml / xlm_to_xlsr
  Official implementation of the paper "Distilling a Pretrained Language Model to a Multilingual ASR Model" (Interspeech 2022)
  ☆12Updated last year
• SmerkyG / RWKV_Explained
  RWKV, in easy to read code
  ☆72Updated 10 months ago
• erogol / BlaGPT
  Experimental playground for benchmarking language model (LM) architectures, layers, and tricks on smaller datasets. Designed for flexible…
  ☆98Updated 2 weeks ago
• nikhilvyas / SOAP
  ☆250Updated last year
• fkodom / yet-another-retnet
  A simple but robust PyTorch implementation of RetNet from "Retentive Network: A Successor to Transformer for Large Language Models" (http…
  ☆106Updated 2 years ago
• HazyResearch / flash-fft-conv
  FlashFFTConv: Efficient Convolutions for Long Sequences with Tensor Cores
  ☆341Updated last year
• DBraun / DAC-JAX
  JAX Implementations of Descript Audio Codec and EnCodec
  ☆33Updated 10 months ago
• kaistmm / voxsim_trainer
  [INTERSPEECH 2024] Official code for VoxSim: A perceptual voice similarity dataset
  ☆12Updated 4 months ago
• proger / hippogriff
  Griffin MQA + Hawk Linear RNN Hybrid
  ☆88Updated last year
• pbelcak / fastfeedforward
  A repository for log-time feedforward networks
  ☆224Updated last year
• ctlllll / SGConv
  ☆163Updated 3 years ago
• cloneofsimo / ezmup
  Simple implementation of muP, based on Spectral Condition for Feature Learning. The implementation is SGD only, dont use it for Adam
  ☆85Updated last year
• nanowell / AdEMAMix-Optimizer-Pytorch
  The AdEMAMix Optimizer: Better, Faster, Older.
  ☆186Updated last year
• AtlasAnalyticsLab / AdaFisher
  [ICLR 2025] AdaFisher: Adaptive Second Order Optimization via Fisher Information
  ☆51Updated last year
• EdwardDixon / snake
  Inspired by "Neural Networks Fail to Learn Periodic Functions and How to Fix It"
  ☆73Updated 7 months ago
• i404788 / s5-pytorch
  Pytorch implementation of Simplified Structured State-Spaces for Sequence Modeling (S5)
  ☆82Updated last year
• xinan-chen / AP_BWE
  Towards High-Quality and Efficient Speech Bandwidth Extension with Parallel Amplitude and Phase Prediction
  ☆13Updated last year
• KdaiP / conformer-RoPE
  Conformer block with Rotary Position Embedding, modified from lucidrains' implement
  ☆16Updated last year
• NX-AI / mlstm_kernels
  Tiled Flash Linear Attention library for fast and efficient mLSTM Kernels.
  ☆84Updated 2 months ago
• lxxue / prefix_sum
  A PyTorch wrapper of parallel exclusive scan in CUDA
  ☆12Updated 2 years ago