Alternatives and similar repositories for FANformer

Users that are interested in FANformer are comparing it to the libraries listed below

• AwesomeSeq / Comba-triton
  ☆15Updated last week
• kyegomez / Griffin
  Implementation of Griffin from the paper: "Griffin: Mixing Gated Linear Recurrences with Local Attention for Efficient Language Models"
  ☆55Updated 2 months ago
• nikhilsab / LLMFE
  This is the official repo for the paper "LLM-FE"
  ☆12Updated 3 weeks ago
• nanowell / Q-Sparse-LLM
  My Implementation of Q-Sparse: All Large Language Models can be Fully Sparsely-Activated
  ☆32Updated 10 months ago
• piotrpiekos / MoSA
  User-friendly implementation of the Mixture-of-Sparse-Attention (MoSA). MoSA selects distinct tokens for each head with expert choice rou…
  ☆21Updated last month
• Doraemonzzz / hgru2-pytorch
  ☆23Updated 9 months ago
• assafbk / DeciMamba
  DeciMamba: Exploring the Length Extrapolation Potential of Mamba (ICLR 2025)
  ☆28Updated 2 months ago
• efficientscaling / Z1
  Repo for "Z1: Efficient Test-time Scaling with Code"
  ☆61Updated 2 months ago
• OpenNLPLab / HGRN
  [NeurIPS 2023 spotlight] Official implementation of HGRN in our NeurIPS 2023 paper - Hierarchically Gated Recurrent Neural Network for Se…
  ☆65Updated last year
• HazyResearch / prefix-linear-attention
  ☆55Updated 11 months ago
• IBM / larimar
  Code for ICML 2024 paper
  ☆26Updated last month
• krafton-ai / mambaformer-icl
  MambaFormer in-context learning experiments and implementation for https://arxiv.org/abs/2402.04248
  ☆55Updated last year
• VITA-Group / WeLore
  From GaLore to WeLore: How Low-Rank Weights Non-uniformly Emerge from Low-Rank Gradients. Ajay Jaiswal, Lu Yin, Zhenyu Zhang, Shiwei Liu,…
  ☆47Updated 2 months ago
• qiuzh20 / gated_attention
  The official implementation for Gated Attention for Large Language Models: Non-linearity, Sparsity, and Attention-Sink-Free
  ☆44Updated last month
• tanaymeh / mamba-train
  A single repo with all scripts and utils to train / fine-tune the Mamba model with or without FIM
  ☆55Updated last year
• dangxingyu / rnn-icrag
  Official repository of paper "RNNs Are Not Transformers (Yet): The Key Bottleneck on In-context Retrieval"
  ☆27Updated last year
• abdelfattah-lab / SplitReason
  ☆16Updated last week
• THUDM / T1
  RL Scaling and Test-Time Scaling (ICML'25)
  ☆106Updated 5 months ago
• NJUNLP / MoE-LPR
  ☆14Updated 6 months ago
• princeton-pli / MeCo
  Code for preprint "Metadata Conditioning Accelerates Language Model Pre-training (MeCo)"
  ☆39Updated last month
• kyegomez / xLSTM
  Implementation of xLSTM in Pytorch from the paper: "xLSTM: Extended Long Short-Term Memory"
  ☆119Updated 2 months ago
• RobertCsordas / moeut
  ☆79Updated 10 months ago
• fe1ixxu / CPO_SIMPO
  This repository contains the joint use of CPO and SimPO method for better reference-free preference learning methods.
  ☆53Updated 10 months ago
• OpenMOSE / RWKV-Infer
  A large-scale RWKV v6, v7(World, PRWKV, Hybrid-RWKV) inference. Capable of inference by combining multiple states(Pseudo MoE). Easy to de…
  ☆38Updated 3 weeks ago
• liujch1998 / ppo-mcts
  ☆13Updated last year
• Doraemonzzz / hgru-pytorch
  ☆27Updated 11 months ago
• SeunghyunSEO / optimized_hf_llama_class_for_training
  ☆47Updated 9 months ago
• kazuki-irie / kv-memory-brain
  Official Code Repository for the paper "Key-value memory in the brain"
  ☆26Updated 4 months ago
• Bui1dMySea / MemLong
  ☆94Updated 6 months ago
• Jiahao004 / DeepTheorem
  ☆19Updated 2 weeks ago