Alternatives and similar repositories for layer_norm_expressivity_role:

Users that are interested in layer_norm_expressivity_role are comparing it to the libraries listed below

• RobertCsordas / linear_layer_as_attention
  The official repository for our paper "The Dual Form of Neural Networks Revisited: Connecting Test Time Predictions to Training Patterns …
  ☆16Updated last year
• gregorbachmann / scaling_mlps
  ☆51Updated 10 months ago
• ahennequ / pytorch-custom-mma
  ☆29Updated 2 years ago
• dangxingyu / rnn-icrag
  Official repository of paper "RNNs Are Not Transformers (Yet): The Key Bottleneck on In-context Retrieval"
  ☆26Updated last year
• shreyansh26 / Attention-Mask-Patterns
  Using FlexAttention to compute attention with different masking patterns
  ☆43Updated 7 months ago
• IBM / ColPret
  Efficient Scaling laws and collaborative pretraining.
  ☆16Updated 2 months ago
• aniquetahir / JORA
  JORA: JAX Tensor-Parallel LoRA Library (ACL 2024)
  ☆33Updated last year
• smonsays / hypernetwork-attention
  Official code for the paper "Attention as a Hypernetwork"
  ☆28Updated 10 months ago
• yikangshen / megablocks
  ☆20Updated 10 months ago
• facebookresearch / coocmap
  code for paper "Accessing higher dimensions for unsupervised word translation"
  ☆21Updated last year
• lucidrains / token-shift-gpt
  Implementation of Token Shift GPT - An autoregressive model that solely relies on shifting the sequence space for mixing
  ☆48Updated 3 years ago
• bdusell / stack-attention
  Code for the paper "Stack Attention: Improving the Ability of Transformers to Model Hierarchical Patterns"
  ☆17Updated last year
• facebookresearch / SIMAT
  codebase for the SIMAT dataset and evaluation
  ☆39Updated 3 years ago
• CyndxAI / QKNorm
  Code for the paper "Query-Key Normalization for Transformers"
  ☆39Updated 4 years ago
• lucidrains / einops-exts
  Implementation of some personal helper functions for Einops, my most favorite tensor manipulation library ❤️
  ☆54Updated 2 years ago
• HazyResearch / prefix-linear-attention
  ☆53Updated 9 months ago
• belindal / LaMPP
  Code for LaMPP: Language Models as Probabilistic Priors for Perception and Action
  ☆36Updated 2 years ago
• VITA-Group / SSM-Bottleneck
  [ICLR'25] "Understanding Bottlenecks of State Space Models through the Lens of Recency and Over-smoothing" by Peihao Wang, Ruisi Cai, Yue…
  ☆11Updated last month
• Lightning-AI / forked-pdb
  Python pdb for multiple processes
  ☆39Updated 2 years ago
• lucidrains / tableformer-pytorch
  Implementation of TableFormer, Robust Transformer Modeling for Table-Text Encoding, in Pytorch
  ☆37Updated 3 years ago
• shawntan / stickbreaking-attention
  Stick-breaking attention
  ☆52Updated last month
• lucidrains / autoregressive-linear-attention-cuda
  CUDA implementation of autoregressive linear attention, with all the latest research findings
  ☆44Updated last year
• Parallel-Reasoning / APR
  Code for Paper: Learning Adaptive Parallel Reasoning with Language Models
  ☆51Updated this week
• epfml / schedules-and-scaling
  Code for NeurIPS 2024 Spotlight: "Scaling Laws and Compute-Optimal Training Beyond Fixed Training Durations"
  ☆71Updated 5 months ago
• EleutherAI / rnngineering
  Engineering the state of RNN language models (Mamba, RWKV, etc.)
  ☆32Updated 11 months ago
• srush / mamba-primer
  ☆37Updated last year
• facebookresearch / ViP-MAE
  This is a PyTorch implementation of the paperViP A Differentially Private Foundation Model for Computer Vision
  ☆36Updated last year
• facebookresearch / iclmlp
  Experiments for "A Closer Look at In-Context Learning under Distribution Shifts"
  ☆20Updated last year
• sustcsonglin / gated_linear_attention_layer
  ☆32Updated last year
• kyegomez / Reka-Torch
  Implementation of the model: "Reka Core, Flash, and Edge: A Series of Powerful Multimodal Language Models" in PyTorch
  ☆30Updated this week