Alternatives and similar repositories for mamba

Users that are interested in mamba are comparing it to the libraries listed below

• krafton-ai / mambaformer-icl
  MambaFormer in-context learning experiments and implementation for https://arxiv.org/abs/2402.04248
  ☆54Updated 11 months ago
• OpenNLPLab / HGRN2
  HGRN2: Gated Linear RNNs with State Expansion
  ☆54Updated 9 months ago
• Adamdad / rational_kat_cu
  ☆61Updated 4 months ago
• assafbk / DeciMamba
  DeciMamba: Exploring the Length Extrapolation Potential of Mamba (ICLR 2025)
  ☆28Updated last month
• kyegomez / SimpleMamba
  Implementation of a modular, high-performance, and simplistic mamba for high-speed applications
  ☆35Updated 6 months ago
• Doraemonzzz / hgru2-pytorch
  ☆23Updated 8 months ago
• fangyuan-ksgk / selective-attention-transformer
  Unofficial Implementation of Selective Attention Transformer
  ☆16Updated 7 months ago
• test-time-training / ttt-lm-kernels
  Inference Speed Benchmark for Learning to (Learn at Test Time): RNNs with Expressive Hidden States
  ☆67Updated 10 months ago
• KindXiaoming / physics_of_skill_learning
  We study toy models of skill learning.
  ☆28Updated 4 months ago
• smonsays / hypernetwork-attention
  Official code for the paper "Attention as a Hypernetwork"
  ☆36Updated 11 months ago
• WailordHe / DenseSSM
  A repository for DenseSSMs
  ☆87Updated last year
• Itamarzimm / UnifiedImplicitAttnRepr
  [ICLR 2025] Official Code Release for Explaining Modern Gated-Linear RNNs via a Unified Implicit Attention Formulation
  ☆42Updated 3 months ago
• nreHieW / minVAR
  Minimal Implementation of Visual Autoregressive Modelling (VAR)
  ☆33Updated 2 months ago
• SHI-Labs / CompactNet
  ☆31Updated last year
• TRI-ML / linear_open_lm
  A repository for research on medium sized language models.
  ☆76Updated last year
• kyegomez / Griffin
  Implementation of Griffin from the paper: "Griffin: Mixing Gated Linear Recurrences with Local Attention for Efficient Language Models"
  ☆55Updated 2 months ago
• kyleliang919 / Online-Subspace-Descent
  This repo is based on https://github.com/jiaweizzhao/GaLore
  ☆28Updated 8 months ago
• RobertCsordas / moeut
  ☆79Updated 9 months ago
• BlinkDL / LinearAttentionArena
  Here we will test various linear attention designs.
  ☆59Updated last year
• RobertCsordas / moe_attention
  Official repository for the paper "SwitchHead: Accelerating Transformers with Mixture-of-Experts Attention"
  ☆97Updated 8 months ago
• kiddyboots216 / lottery-ticket-adaptation
  Lottery Ticket Adaptation
  ☆39Updated 6 months ago
• kyegomez / MultiQueryAttention
  This is a simple torch implementation of the high performance Multi-Query Attention
  ☆17Updated last year
• ml-jku / EVA
  One Initialization to Rule them All: Fine-tuning via Explained Variance Adaptation
  ☆40Updated 7 months ago
• piotrpiekos / MoSA
  User-friendly implementation of the Mixture-of-Sparse-Attention (MoSA). MoSA selects distinct tokens for each head with expert choice rou…
  ☆17Updated last month
• GATECH-EIC / Linearized-LLM
  [ICML 2024] When Linear Attention Meets Autoregressive Decoding: Towards More Effective and Efficient Linearized Large Language Models
  ☆30Updated 11 months ago
• ElleLeonne / Lightning-ReLoRA
  A public implementation of the ReLoRA pretraining method, built on Lightning-AI's Pytorch Lightning suite.
  ☆33Updated last year
• IST-DASLab / QuEST
  Work in progress.
  ☆68Updated last week
• kyegomez / Mixture-of-Depths
  Implementation of the paper: "Mixture-of-Depths: Dynamically allocating compute in transformer-based language models"
  ☆94Updated this week
• zqOuO / GWT
  ☆13Updated 4 months ago
• catid / spectral_ssm
  Implementation of Spectral State Space Models
  ☆16Updated last year