Alternatives and similar repositories for gated_attention

Users that are interested in gated_attention are comparing it to the libraries listed below

• kyegomez / SwitchTransformers
  Implementation of Switch Transformers from the paper: "Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficien…
  ☆135Updated 2 months ago
• ApexGen-X / MergeVQ
  [CVPR'25] MergeVQ: A Unified Framework for Visual Generation and Representation with Token Merging and Quantization
  ☆47Updated 5 months ago
• tensorgi / TPA
  [NeurIPS 2025 Spotlight] TPA: Tensor ProducT ATTenTion Transformer (T6) (https://arxiv.org/abs/2501.06425)
  ☆438Updated last week
• RobinWu218 / ToST
  [ICLR 2025 Spotlight] Official Implementation for ToST (Token Statistics Transformer)
  ☆128Updated 10 months ago
• TsinghuaC3I / Fourier-Position-Embedding
  [ICML 2025] Fourier Position Embedding: Enhancing Attention’s Periodic Extension for Length Generalization
  ☆105Updated 6 months ago
• Clin0212 / HydraLoRA
  [NeurIPS'24 Oral] HydraLoRA: An Asymmetric LoRA Architecture for Efficient Fine-Tuning
  ☆234Updated last year
• LMM101 / Awesome-Multimodal-Next-Token-Prediction
  [Survey] Next Token Prediction Towards Multimodal Intelligence: A Comprehensive Survey
  ☆465Updated 11 months ago
• ML-GSAI / LLaDA-V
  ☆301Updated last week
• transformer-vq / transformer_vq
  ☆200Updated 2 years ago
• lucidrains / native-sparse-attention-pytorch
  Implementation of the sparse attention pattern proposed by the Deepseek team in their "Native Sparse Attention" paper
  ☆790Updated 4 months ago
• hkproj / mamba-notes
  Notes on the Mamba and the S4 model (Mamba: Linear-Time Sequence Modeling with Selective State Spaces)
  ☆175Updated last year
• pengzhangzhi / Open-dLLM
  The most open diffusion language model for code generation — releasing pretraining, evaluation, inference, and checkpoints.
  ☆490Updated last month
• NVlabs / GatedDeltaNet
  [ICLR 2025] Official PyTorch Implementation of Gated Delta Networks: Improving Mamba2 with Delta Rule
  ☆404Updated 3 months ago
• hanyang1999 / discrete-diffusion-papers
  A collection of papers on discrete diffusion models
  ☆167Updated 5 months ago
• yczhou001 / Awesome-Diffusion-LLM
  paper list, tutorial, and nano code snippet for Diffusion Large Language Models.
  ☆148Updated 6 months ago
• MuLabPKU / TransMLA
  TransMLA: Multi-Head Latent Attention Is All You Need (NeurIPS 2025 Spotlight)
  ☆419Updated 3 months ago
• naver-ai / rope-vit
  [ECCV 2024] Official PyTorch implementation of RoPE-ViT "Rotary Position Embedding for Vision Transformer"
  ☆429Updated last month
• zhijie-group / Discrete-Diffusion-Forcing
  Discrete Diffusion Forcing (D2F): dLLMs Can Do Faster-Than-AR Inference
  ☆216Updated 3 months ago
• Caiyun-AI / MUDDFormer
  ☆85Updated 7 months ago
• AmeenAli / HiddenMambaAttn
  Official PyTorch Implementation of "The Hidden Attention of Mamba Models"
  ☆232Updated 2 months ago
• Hprairie / Bi-Mamba2
  A Triton Kernel for incorporating Bi-Directionality in Mamba2
  ☆75Updated last year
• DAMO-NLP-SG / Inf-CLIP
  [CVPR 2025 Highlight] The official CLIP training codebase of Inf-CL: "Breaking the Memory Barrier: Near Infinite Batch Size Scaling for C…
  ☆275Updated 11 months ago
• VILA-Lab / Awesome-DLMs
  The official GitHub repo for the survey paper "A Survey on Diffusion Language Models".
  ☆600Updated this week
• radarFudan / Awesome-state-space-models
  Collection of papers on state-space models
  ☆613Updated last month
• lucidrains / autoregressive-diffusion-pytorch
  Implementation of Autoregressive Diffusion in Pytorch
  ☆430Updated 3 weeks ago
• zitian-gao / one-shot-em
  One-shot Entropy Minimization
  ☆187Updated 6 months ago
• Outsider565 / LoRA-GA
  ☆216Updated last month
• yongliang-wu / DFT
  [Preprint] On the Generalization of SFT: A Reinforcement Learning Perspective with Reward Rectification.
  ☆513Updated last month
• LINs-lab / DynMoE
  [ICLR 2025] Dynamic Mixture of Experts: An Auto-Tuning Approach for Efficient Transformer Models
  ☆152Updated 5 months ago
• kyleliang919 / C-Optim
  When it comes to optimizers, it's always better to be safe than sorry
  ☆397Updated 3 months ago