Alternatives and similar repositories for top_k_attention

Users that are interested in top_k_attention are comparing it to the libraries listed below

• subho406 / agalite

  View on GitHub
  AGaLiTe: Approximate Gated Linear Transformers for Online Reinforcement Learning (Published in TMLR)
  ☆23Oct 15, 2024Updated last year
• UniversalDependencies / UD_German-HDT

  View on GitHub
  ☆13Nov 28, 2025Updated 2 months ago
• nlpai-lab / Korean-CommonGen

  View on GitHub
  [Findings of NAACL2022] A Dog Is Passing Over The Jet? A Text-Generation Dataset for Korean Commonsense Reasoning and Evaluation
  ☆11May 27, 2022Updated 3 years ago
• RobertCsordas / ndr

  View on GitHub
  The official repository for our paper "The Neural Data Router: Adaptive Control Flow in Transformers Improves Systematic Generalization".
  ☆34Jun 11, 2025Updated 8 months ago
• teffland / ner-expected-entity-ratio

  View on GitHub
  Implementation and experiments for Partially Supervised NER via Expected Entity Ratio in TACL 2022
  ☆14Nov 7, 2022Updated 3 years ago
• radarFudan / Curse-of-memory

  View on GitHub
  Curse-of-memory phenomenon of RNNs in sequence modelling
  ☆19May 8, 2025Updated 9 months ago
• google-deepmind / spectral_ssm

  View on GitHub
  ☆35Apr 12, 2024Updated last year
• yikangshen / megablocks

  View on GitHub
  ☆20May 30, 2024Updated last year
• OpenNLPLab / HGRN2

  View on GitHub
  HGRN2: Gated Linear RNNs with State Expansion
  ☆56Aug 20, 2024Updated last year
• maximzubkov / fft-scan

  View on GitHub
  Efficient PScan implementation in PyTorch
  ☆17Jan 2, 2024Updated 2 years ago
• thunlp / SparsingLaw

  View on GitHub
  The open-source materials for paper "Sparsing Law: Towards Large Language Models with Greater Activation Sparsity".
  ☆30Nov 12, 2024Updated last year
• mlpc-ucsd / BERT_Convolutions

  View on GitHub
  (ACL-IJCNLP 2021) Convolutions and Self-Attention: Re-interpreting Relative Positions in Pre-trained Language Models.
  ☆21Jul 13, 2022Updated 3 years ago
• davidsvy / cosformer-pytorch

  View on GitHub
  Unofficial PyTorch implementation of the paper "cosFormer: Rethinking Softmax In Attention".
  ☆44Oct 29, 2021Updated 4 years ago
• fla-org / fla-zoo

  View on GitHub
  Flash-Linear-Attention models beyond language
  ☆21Aug 28, 2025Updated 5 months ago
• juvi21 / CoPE-cuda

  View on GitHub
  Contextual Position Encoding but with some custom CUDA Kernels https://arxiv.org/abs/2405.18719
  ☆22Jun 5, 2024Updated last year
• mlpen / YOSO

  View on GitHub
  ☆21Jul 1, 2021Updated 4 years ago
• Aleph-Alpha-Research / NeurIPS-WANT-submission-efficient-parallelization-layouts

  View on GitHub
  ☆22Dec 15, 2023Updated 2 years ago
• Doraemonzzz / hgru2-pytorch

  View on GitHub
  ☆24Sep 25, 2024Updated last year
• tml-epfl / why-weight-decay

  View on GitHub
  Why Do We Need Weight Decay in Modern Deep Learning? [NeurIPS 2024]
  ☆70Sep 25, 2024Updated last year
• glassroom / heinsen_attention

  View on GitHub
  Reference implementation of "Softmax Attention with Constant Cost per Token" (Heinsen, 2024)
  ☆24Jun 6, 2024Updated last year
• sail-sg / TEC

  View on GitHub
  ☆17Mar 17, 2023Updated 2 years ago
• LuisaMaerz / KnowMAN

  View on GitHub
  KnowMAN: Weakly Supervised Multinomial Adversarial Networks
  ☆12Nov 9, 2021Updated 4 years ago
• Ryu1845 / hyena-jax

  View on GitHub
  Implementation of Hyena Hierarchy in JAX
  ☆10Apr 30, 2023Updated 2 years ago
• VITA-Group / SSM-Bottleneck

  View on GitHub
  [ICLR'25] "Understanding Bottlenecks of State Space Models through the Lens of Recency and Over-smoothing" by Peihao Wang, Ruisi Cai, Yue…
  ☆17Mar 21, 2025Updated 10 months ago
• Doraemonzzz / nanoTransNormer

  View on GitHub
  ☆11Oct 11, 2023Updated 2 years ago
• HazyResearch / embroid

  View on GitHub
  Embroid: Unsupervised Prediction Smoothing Can Improve Few-Shot Classification
  ☆11Aug 12, 2023Updated 2 years ago
• Yuanhy1997 / HyPe

  View on GitHub
  HyPe: Better Pre-trained Language Model Fine-tuning with Hidden Representation Perturbation [ACL 2023]
  ☆14Jul 11, 2023Updated 2 years ago
• Noahs-ARK / PaLM

  View on GitHub
  PyTorch implementation for PaLM: A Hybrid Parser and Language Model.
  ☆10Jan 7, 2020Updated 6 years ago
• rycolab / aflt-f2023

  View on GitHub
  Advanced Formal Language Theory (263-5352-00L; Frühjahr 2023)
  ☆10Feb 21, 2023Updated 2 years ago
• SpeechColab / PySpeechColab

  View on GitHub
  A library of speech gadgets.
  ☆14Oct 15, 2022Updated 3 years ago
• Doraemonzzz / hgru-pytorch

  View on GitHub
  ☆29Jul 9, 2024Updated last year
• frankaging / Causal-Distill

  View on GitHub
  The Codebase for Causal Distillation for Language Models (NAACL '22)
  ☆26May 1, 2022Updated 3 years ago
• g8a9 / ear

  View on GitHub
  Code associated with the paper "Entropy-based Attention Regularization Frees Unintended Bias Mitigation from Lists"
  ☆50May 31, 2022Updated 3 years ago
• ahennequ / pytorch-custom-mma

  View on GitHub
  ☆29Oct 3, 2022Updated 3 years ago
• dangxingyu / rnn-icrag

  View on GitHub
  Official repository of paper "RNNs Are Not Transformers (Yet): The Key Bottleneck on In-context Retrieval"
  ☆27Apr 17, 2024Updated last year
• habanero-lab / APPy

  View on GitHub
  APPy (Annotated Parallelism for Python) enables users to annotate loops and tensor expressions in Python with compiler directives akin to…
  ☆30Jan 28, 2026Updated 2 weeks ago
• 10-zin / Synthesizer

  View on GitHub
  A PyTorch implementation of the paper - "Synthesizer: Rethinking Self-Attention in Transformer Models"
  ☆73Dec 8, 2022Updated 3 years ago
• RUCAIBox / MPOP

  View on GitHub
  ☆13Jun 16, 2021Updated 4 years ago
• TalSchuster / CATs

  View on GitHub
  Confident Adaptive Transformers
  ☆14Apr 18, 2021Updated 4 years ago