CyndxAI / QKNorm

Related projects ⓘ

Alternatives and complementary repositories for QKNorm

• mlpc-ucsd / BERT_Convolutions
  (ACL-IJCNLP 2021) Convolutions and Self-Attention: Re-interpreting Relative Positions in Pre-trained Language Models.
  ☆21Updated 2 years ago
• lucidrains / coco-lm-pytorch
  Implementation of COCO-LM, Correcting and Contrasting Text Sequences for Language Model Pretraining, in Pytorch
  ☆45Updated 3 years ago
• pkuzengqi / Skyformer
  Skyformer: Remodel Self-Attention with Gaussian Kernel and Nystr\"om Method (NeurIPS 2021)
  ☆59Updated 2 years ago
• Noahs-ARK / RFA
  ☆32Updated 3 years ago
• lucidrains / AoA-pytorch
  A Pytorch implementation of Attention on Attention module (both self and guided variants), for Visual Question Answering
  ☆40Updated 4 years ago
• RobertCsordas / ndr
  The official repository for our paper "The Neural Data Router: Adaptive Control Flow in Transformers Improves Systematic Generalization".
  ☆32Updated 3 years ago
• lucidrains / learning-to-expire-pytorch
  An implementation of Transformer with Expire-Span, a circuit for learning which memories to retain
  ☆33Updated 4 years ago
• yikangshen / megablocks
  ☆18Updated 5 months ago
• antofuller / configaformers
  A python library for highly configurable transformers - easing model architecture search and experimentation.
  ☆49Updated 2 years ago
• RobertCsordas / linear_layer_as_attention
  The official repository for our paper "The Dual Form of Neural Networks Revisited: Connecting Test Time Predictions to Training Patterns …
  ☆16Updated last year
• intersun / CoDIR
  Code for EMNLP 2020 paper CoDIR
  ☆41Updated 2 years ago
• sustcsonglin / gated_linear_attention_layer
  ☆31Updated 10 months ago
• ischlag / Fast-Weight-Memory-public
  Official code repository of the paper Learning Associative Inference Using Fast Weight Memory by Schlag et al.
  ☆26Updated 3 years ago
• schwartz-lab-NLP / papa
  Code for the PAPA paper
  ☆27Updated 2 years ago
• lucidrains / distilled-retriever-pytorch
  Implementation of the retriever distillation procedure as outlined in the paper "Distilling Knowledge from Reader to Retriever"
  ☆32Updated 3 years ago
• lucidrains / token-shift-gpt
  Implementation of Token Shift GPT - An autoregressive model that solely relies on shifting the sequence space for mixing
  ☆47Updated 2 years ago
• thu-ml / LM-Calibration
  ☆15Updated last year
• sarthmit / Compositional-Attention
  Code to reproduce the results for Compositional Attention
  ☆60Updated 2 years ago
• cpcp1998 / PermuteFormer
  Code for the paper PermuteFormer
  ☆42Updated 3 years ago
• lavoiems / simplicial-embeddings
  solo-learn: a library of self-supervised methods for visual representation learning powered by Pytorch Lightning
  ☆23Updated last year
• yxuansu / Contrastive_Search_versus_Contrastive_Decoding
  An Empirical Study On Contrastive Search And Contrastive Decoding For Open-ended Text Generation
  ☆26Updated 5 months ago
• iedwardwangi / MetaAdapter
  ☆22Updated 3 years ago
• yxuansu / Awesome_Diffusions
  ☆16Updated last year
• haorannlp / mix
  Code for "Mixed Cross Entropy Loss for Neural Machine Translation"
  ☆20Updated 3 years ago
• HA-Transformer / MAT
  The implementation of multi-branch attentive Transformer (MAT).
  ☆33Updated 4 years ago
• cheneydon / efficient-bert
  This repository contains the code for the paper in Findings of EMNLP 2021: "EfficientBERT: Progressively Searching Multilayer Perceptron …
  ☆32Updated last year
• RobertCsordas / transformer_generalization
  The official repository for our paper "The Devil is in the Detail: Simple Tricks Improve Systematic Generalization of Transformers". We s…
  ☆66Updated last year
• jenni-ai / T2FW
  Fine-Tuning Pre-trained Transformers into Decaying Fast Weights
  ☆19Updated 2 years ago
• gmftbyGMFTBY / MomentumDecoding
  Momentum Decoding: Open-ended Text Generation as Graph Exploration
  ☆19Updated last year
• ahennequ / pytorch-custom-mma
  ☆29Updated 2 years ago