Alternatives and similar repositories for QKNorm:

Users that are interested in QKNorm are comparing it to the libraries listed below

• lucidrains / coco-lm-pytorch
  Implementation of COCO-LM, Correcting and Contrasting Text Sequences for Language Model Pretraining, in Pytorch
  ☆45Updated 4 years ago
• mlpc-ucsd / BERT_Convolutions
  (ACL-IJCNLP 2021) Convolutions and Self-Attention: Re-interpreting Relative Positions in Pre-trained Language Models.
  ☆21Updated 2 years ago
• RobertCsordas / linear_layer_as_attention
  The official repository for our paper "The Dual Form of Neural Networks Revisited: Connecting Test Time Predictions to Training Patterns …
  ☆16Updated last year
• yikangshen / megablocks
  ☆19Updated 10 months ago
• lucidrains / token-shift-gpt
  Implementation of Token Shift GPT - An autoregressive model that solely relies on shifting the sequence space for mixing
  ☆48Updated 3 years ago
• romebert / RomeBERT
  ☆16Updated 3 years ago
• ahennequ / pytorch-custom-mma
  ☆29Updated 2 years ago
• lucidrains / tableformer-pytorch
  Implementation of TableFormer, Robust Transformer Modeling for Table-Text Encoding, in Pytorch
  ☆37Updated 3 years ago
• microsoft / AMOS
  [ICLR 2022] Pretraining Text Encoders with Adversarial Mixture of Training Signal Generators
  ☆24Updated last year
• antofuller / configaformers
  A python library for highly configurable transformers - easing model architecture search and experimentation.
  ☆49Updated 3 years ago
• MikeWangWZHL / Zemi
  Repo for "Zemi: Learning Zero-Shot Semi-Parametric Language Models from Multiple Tasks" ACL 2023 Findings
  ☆16Updated last year
• ischlag / Fast-Weight-Memory-public
  Official code repository of the paper Learning Associative Inference Using Fast Weight Memory by Schlag et al.
  ☆28Updated 4 years ago
• JunShern / few-shot-adaptation
  Exploring Few-Shot Adaptation of Language Models with Tables
  ☆23Updated 2 years ago
• thunlp / DPT
  ☆13Updated 2 years ago
• frankaging / Reason-SCAN
  ReaSCAN is a synthetic navigation task that requires models to reason about surroundings over syntactically difficult languages. (NeurIPS…
  ☆20Updated 3 years ago
• jenni-ai / T2FW
  Fine-Tuning Pre-trained Transformers into Decaying Fast Weights
  ☆19Updated 2 years ago
• lucidrains / panoptic-transformer
  Another attempt at a long-context / efficient transformer by me
  ☆37Updated 3 years ago
• PAL-ML / PEARL_v1
  ☆29Updated 3 years ago
• sustcsonglin / gated_linear_attention_layer
  ☆32Updated last year
• yxuansu / Awesome_Diffusions
  ☆17Updated 2 years ago
• Noahs-ARK / RFA
  ☆33Updated 4 years ago
• pkuzengqi / Skyformer
  Skyformer: Remodel Self-Attention with Gaussian Kernel and Nystr\"om Method (NeurIPS 2021)
  ☆60Updated 2 years ago
• gmftbyGMFTBY / MomentumDecoding
  Momentum Decoding: Open-ended Text Generation as Graph Exploration
  ☆19Updated 2 years ago
• lucidrains / esbn-transformer
  An attempt to merge ESBN with Transformers, to endow Transformers with the ability to emergently bind symbols
  ☆15Updated 3 years ago
• lucidrains / einops-exts
  Implementation of some personal helper functions for Einops, my most favorite tensor manipulation library ❤️
  ☆54Updated 2 years ago
• cpcp1998 / PermuteFormer
  Code for the paper PermuteFormer
  ☆42Updated 3 years ago
• google-deepmind / emergent_in_context_learning
  ☆81Updated 8 months ago
• CAMTL / CA-MTL
  Conditionally Adaptive Multi-Task Learning: Improving Transfer Learning in NLP Using Fewer Parameters & Less Data
  ☆56Updated 3 years ago
• shizhediao / DaVinci
  Source code for the paper "Prefix Language Models are Unified Modal Learners"
  ☆43Updated last year
• schwartz-lab-NLP / papa
  Code for the PAPA paper
  ☆27Updated 2 years ago