Alternatives and similar repositories for selective-attention-transformer

Users that are interested in selective-attention-transformer are comparing it to the libraries listed below

• RobertCsordas / moeut
  ☆79Updated 10 months ago
• ablghtianyi / ICL_Modular_Arithmetic
  ☆19Updated 3 months ago
• pixeli99 / MixLN
  [ICLR 2025] Official Pytorch Implementation of "Mix-LN: Unleashing the Power of Deeper Layers by Combining Pre-LN and Post-LN" by Pengxia…
  ☆24Updated 6 months ago
• kyleliang919 / Online-Subspace-Descent
  This repo is based on https://github.com/jiaweizzhao/GaLore
  ☆28Updated 9 months ago
• kuleshov-group / remdm
  Remasking Discrete Diffusion Models with Inference-Time Scaling
  ☆26Updated 3 months ago
• formll / resolving-scaling-law-discrepancies
  ☆20Updated 11 months ago
• zqOuO / GWT
  ☆13Updated 5 months ago
• TianjinYellow / SPAM-Optimizer
  ☆33Updated 3 months ago
• wang-kee / LiNeS
  Official repository of "LiNeS: Post-training Layer Scaling Prevents Forgetting and Enhances Model Merging"
  ☆29Updated 7 months ago
• ml-jku / EVA
  One Initialization to Rule them All: Fine-tuning via Explained Variance Adaptation
  ☆40Updated 8 months ago
• OpenNLPLab / HGRN2
  HGRN2: Gated Linear RNNs with State Expansion
  ☆55Updated 10 months ago
• VijayLingam95 / SVFT
  ☆29Updated 4 months ago
• shikaiqiu / compute-better-spent
  ☆53Updated 8 months ago
• zhixuan-lin / forgetting-transformer
  [ICLR 2025] Official PyTorch implementation of "Forgetting Transformer: Softmax Attention with a Forget Gate"
  ☆108Updated last month
• dayal-kalra / low-memory-adam
  ☆12Updated 3 months ago
• katiekang1998 / reasoning_generalization
  ☆32Updated 5 months ago
• smonsays / hypernetwork-attention
  Official code for the paper "Attention as a Hypernetwork"
  ☆39Updated last year
• assafbk / DeciMamba
  DeciMamba: Exploring the Length Extrapolation Potential of Mamba (ICLR 2025)
  ☆28Updated 2 months ago
• Bond1995 / Markov
  Code for experiments on transformers using Markovian data.
  ☆15Updated 7 months ago
• JeanKaddour / LAWA
  Latest Weight Averaging (NeurIPS HITY 2022)
  ☆30Updated 2 years ago
• Red-Hat-AI-Innovation-Team / SQuat
  ☆13Updated 3 weeks ago
• Benjamin-Walker / selective-ssms-and-linear-cdes
  Code for "Theoretical Foundations of Deep Selective State-Space Models" (NeurIPS 2024)
  ☆15Updated 5 months ago
• fal-ai-community / nano-mdm
  Tiny re-implementation of MDM in style of LLaDA and nano-gpt speedrun
  ☆52Updated 3 months ago
• KindXiaoming / physics_of_skill_learning
  We study toy models of skill learning.
  ☆28Updated 5 months ago
• tml-epfl / why-weight-decay
  Why Do We Need Weight Decay in Modern Deep Learning? [NeurIPS 2024]
  ☆66Updated 9 months ago
• oripress / EntropyEnigma
  Official code for the ICML 2024 paper "The Entropy Enigma: Success and Failure of Entropy Minimization"
  ☆52Updated last year
• dmis-lab / Monet
  [ICLR 2025] Monet: Mixture of Monosemantic Experts for Transformers
  ☆69Updated this week
• Laz4rz / mup
  Minimal (truly) muP implementation, consistent with TP4 and TP5 papers notation
  ☆14Updated last month
• IdoAmos / not-from-scratch
  ☆31Updated 8 months ago
• sail-sg / SkyLadder
  The official repository for SkyLadder: Better and Faster Pretraining via Context Window Scheduling
  ☆32Updated 3 months ago