Alternatives and similar repositories for mamba4transformers

Users that are interested in mamba4transformers are comparing it to the libraries listed below

• kyegomez / MoE-Mamba
  Implementation of MoE Mamba from the paper: "MoE-Mamba: Efficient Selective State Space Models with Mixture of Experts" in Pytorch and Ze…
  ☆107Updated 2 months ago
• WailordHe / DenseSSM
  A repository for DenseSSMs
  ☆87Updated last year
• chuanyang-Zheng / DAPE
  The this is the official implementation of "DAPE: Data-Adaptive Positional Encoding for Length Extrapolation"
  ☆38Updated 8 months ago
• kyegomez / Mixture-of-Depths
  Implementation of the paper: "Mixture-of-Depths: Dynamically allocating compute in transformer-based language models"
  ☆95Updated 2 weeks ago
• assafbk / DeciMamba
  DeciMamba: Exploring the Length Extrapolation Potential of Mamba (ICLR 2025)
  ☆28Updated 2 months ago
• zitian-gao / one-shot-em
  One-shot Entropy Minimization
  ☆149Updated 2 weeks ago
• TsinghuaC3I / Fourier-Position-Embedding
  [ICML 2025] Fourier Position Embedding: Enhancing Attention’s Periodic Extension for Length Generalization
  ☆71Updated 3 weeks ago
• dhcode-cpp / NSA-pytorch
  DeepSeek Native Sparse Attention pytorch implementation
  ☆71Updated 3 months ago
• kyegomez / Griffin
  Implementation of Griffin from the paper: "Griffin: Mixing Gated Linear Recurrences with Local Attention for Efficient Language Models"
  ☆55Updated 2 months ago
• ZhenweiAn / Dynamic_MoE
  Inference Code for Paper "Harder Tasks Need More Experts: Dynamic Routing in MoE Models"
  ☆52Updated 10 months ago
• EffiVLM-Bench / EffiVLM-Bench
  ☆16Updated 3 weeks ago
• THUDM / Awesome-Parameter-Efficient-Fine-Tuning-for-Foundation-Models
  Parameter-Efficient Fine-Tuning for Foundation Models
  ☆69Updated 2 months ago
• abdelfattah-lab / SplitReason
  ☆16Updated this week
• BestAnHongjun / SentenceVAE
  Enable Next-sentence Prediction for Large Language Models with Faster Speed, Higher Accuracy and Longer Context
  ☆32Updated 10 months ago
• horseee / CoT-Valve
  CoT-Valve: Length-Compressible Chain-of-Thought Tuning
  ☆73Updated 4 months ago
• sramshetty / mixture-of-depths
  An unofficial implementation of "Mixture-of-Depths: Dynamically allocating compute in transformer-based language models"
  ☆35Updated last year
• AwesomeSeq / Comba-triton
  ☆18Updated last week
• euclid-multimodal / Euclid
  ☆16Updated 5 months ago
• USTC-StarTeam / ZIP
  ☆22Updated 11 months ago
• nishiwen1214 / Benchmark-leakage-detection
  Official completion of “Training on the Benchmark Is Not All You Need”.
  ☆34Updated 5 months ago
• DIRECT-BIT / SRA-MCTS
  ☆33Updated 3 weeks ago
• starrYYxuan / LeCo
  This the implementation of LeCo
  ☆31Updated 5 months ago
• pengzhangzhi / Awesome-Mamba
  Awesome list of papers that extend Mamba to various applications.
  ☆133Updated 2 weeks ago
• OpenSparseLLMs / Linear-MoE
  ☆104Updated 2 weeks ago
• RyanLiu112 / GenPRM
  Official codebase for "GenPRM: Scaling Test-Time Compute of Process Reward Models via Generative Reasoning".
  ☆75Updated 3 weeks ago
• MingyuJ666 / The-Impact-of-Reasoning-Step-Length-on-Large-Language-Models
  [ACL'24] Chain of Thought (CoT) is significant in improving the reasoning abilities of large language models (LLMs). However, the correla…
  ☆45Updated last month
• OpenSparseLLMs / MoM
  ☆85Updated 2 months ago
• ArminAzizi98 / LaMDA
  ☆15Updated 7 months ago
• YangLing0818 / SuperCorrect-llm
  [ICLR 2025] SuperCorrect: Advancing Small LLM Reasoning with Thought Template Distillation and Self-Correction
  ☆72Updated 3 months ago
• yegcjs / DiffusionLLM
  Code for paper "Diffusion Language Models Can Perform Many Tasks with Scaling and Instruction-Finetuning"
  ☆80Updated last year