Alternatives and similar repositories for QKNorm

Users that are interested in QKNorm are comparing it to the libraries listed below

• mlpc-ucsd / BERT_Convolutions
  (ACL-IJCNLP 2021) Convolutions and Self-Attention: Re-interpreting Relative Positions in Pre-trained Language Models.
  ☆21Updated 2 years ago
• RobertCsordas / linear_layer_as_attention
  The official repository for our paper "The Dual Form of Neural Networks Revisited: Connecting Test Time Predictions to Training Patterns …
  ☆16Updated 2 weeks ago
• shizhediao / DaVinci
  Source code for the paper "Prefix Language Models are Unified Modal Learners"
  ☆43Updated 2 years ago
• Noahs-ARK / RFA
  ☆33Updated 4 years ago
• shawnlimn / ScaleGrad
  Source code for ScaleGrad
  ☆18Updated 3 years ago
• gmftbyGMFTBY / MomentumDecoding
  Momentum Decoding: Open-ended Text Generation as Graph Exploration
  ☆19Updated 2 years ago
• bergen / EdgeTransformer
  ☆22Updated 3 years ago
• bigscience-workshop / architecture-objective
  ☆97Updated last year
• yikangshen / megablocks
  ☆20Updated last year
• romebert / RomeBERT
  ☆16Updated 4 years ago
• sunyt32 / torchscale
  Transformers at any scale
  ☆41Updated last year
• princeton-nlp / WhatICLLearns
  [ACL 2023 Findings] What In-Context Learning “Learns” In-Context: Disentangling Task Recognition and Task Learning
  ☆21Updated last year
• ischlag / Fast-Weight-Memory-public
  Official code repository of the paper Learning Associative Inference Using Fast Weight Memory by Schlag et al.
  ☆28Updated 4 years ago
• lucidrains / coco-lm-pytorch
  Implementation of COCO-LM, Correcting and Contrasting Text Sequences for Language Model Pretraining, in Pytorch
  ☆46Updated 4 years ago
• kernelmachine / demix
  DEMix Layers for Modular Language Modeling
  ☆53Updated 3 years ago
• cheneydon / efficient-bert
  This repository contains the code for the paper in Findings of EMNLP 2021: "EfficientBERT: Progressively Searching Multilayer Perceptron …
  ☆33Updated 2 years ago
• Alibaba-NLP / MuVER
  [EMNLP 2021] MuVER: Improving First-Stage Entity Retrieval with Multi-View Entity Representations
  ☆31Updated 3 years ago
• yxuansu / Awesome_Diffusions
  ☆17Updated 2 years ago
• thunlp / DPT
  ☆13Updated 3 years ago
• LooperXX / ManagerTower
  Code for ACL 2023 Oral Paper: ManagerTower: Aggregating the Insights of Uni-Modal Experts for Vision-Language Representation Learning
  ☆11Updated 6 months ago
• LAION-AI / Conditional-Pretraining-of-Large-Language-Models
  ☆37Updated 2 years ago
• zinengtang / Perceiver_VL
  PyTorch code for "Perceiver-VL: Efficient Vision-and-Language Modeling with Iterative Latent Attention" (WACV 2023)
  ☆33Updated 2 years ago
• princeton-nlp / DinkyTrain
  Princeton NLP's pre-training library based on fairseq with DeepSpeed kernel integration 🚃
  ☆114Updated 2 years ago
• renll / SparseLT
  [EMNLP 2022] Language Model Pre-Training with Sparse Latent Typing
  ☆14Updated 2 years ago
• yale-nlp / refdpo
  ☆16Updated 11 months ago
• peterbhase / SLAG-Belief-Updating
  Code for paper "Do Language Models Have Beliefs? Methods for Detecting, Updating, and Visualizing Model Beliefs"
  ☆28Updated 3 years ago
• antofuller / configaformers
  A python library for highly configurable transformers - easing model architecture search and experimentation.
  ☆49Updated 3 years ago
• violet-zct / swarm-distillation-zero-shot
  ☆22Updated 2 years ago
• microsoft / AMOS
  [ICLR 2022] Pretraining Text Encoders with Adversarial Mixture of Training Signal Generators
  ☆24Updated last year
• sustcsonglin / gated_linear_attention_layer
  ☆32Updated last year