tech-srl / layer_norm_expressivity_role
Code for the paper "On the Expressivity Role of LayerNorm in Transformers' Attention" (Findings of ACL'2023)
☆46Updated 6 months ago
Alternatives and similar repositories for layer_norm_expressivity_role:
Users that are interested in layer_norm_expressivity_role are comparing it to the libraries listed below
- Implementation of Token Shift GPT - An autoregressive model that solely relies on shifting the sequence space for mixing☆48Updated 3 years ago
- The official repository for our paper "The Dual Form of Neural Networks Revisited: Connecting Test Time Predictions to Training Patterns …☆16Updated last year
- Implementation of TableFormer, Robust Transformer Modeling for Table-Text Encoding, in Pytorch☆37Updated 3 years ago
- Official repository of paper "RNNs Are Not Transformers (Yet): The Key Bottleneck on In-context Retrieval"☆26Updated 11 months ago
- ☆51Updated 9 months ago
- ☆29Updated 2 years ago
- codebase for the SIMAT dataset and evaluation☆39Updated 3 years ago
- Implementation of some personal helper functions for Einops, my most favorite tensor manipulation library ❤️☆54Updated 2 years ago
- Using FlexAttention to compute attention with different masking patterns☆42Updated 6 months ago
- Official code for the paper "Attention as a Hypernetwork"☆25Updated 9 months ago
- Code for the paper "Stack Attention: Improving the Ability of Transformers to Model Hierarchical Patterns"☆17Updated last year
- Code for the paper "Query-Key Normalization for Transformers"☆38Updated 4 years ago
- Another attempt at a long-context / efficient transformer by me☆37Updated 2 years ago
- ☆18Updated 10 months ago
- Code for the PAPA paper☆27Updated 2 years ago
- ☆37Updated last year
- HGRN2: Gated Linear RNNs with State Expansion☆53Updated 7 months ago
- ☆30Updated last year
- ☆52Updated 8 months ago
- Xmixers: A collection of SOTA efficient token/channel mixers☆11Updated 4 months ago
- PyTorch codes for the paper "An Empirical Study of Multimodal Model Merging"☆38Updated last year
- Remasking Discrete Diffusion Models with Inference-Time Scaling☆16Updated 3 weeks ago
- JORA: JAX Tensor-Parallel LoRA Library (ACL 2024)☆33Updated 11 months ago
- Code for LaMPP: Language Models as Probabilistic Priors for Perception and Action☆36Updated last year
- Here we will test various linear attention designs.☆60Updated 11 months ago
- [ICLR 2025] Code for the paper "Beyond Autoregression: Discrete Diffusion for Complex Reasoning and Planning"☆45Updated last month
- Contains my experiments with the `big_vision` repo to train ViTs on ImageNet-1k.☆22Updated 2 years ago
- code for paper "Accessing higher dimensions for unsupervised word translation"☆21Updated last year
- Unofficial implementation of paper : Exploring the Space of Key-Value-Query Models with Intention☆11Updated last year
- ☆102Updated last year