Alternatives and similar repositories for RWKV-Survey

Users that are interested in RWKV-Survey are comparing it to the libraries listed below

• WailordHe / DenseSSM
  A repository for DenseSSMs
  ☆89Updated last year
• leo-yangli / VB-LoRA
  This repo contains the source code for VB-LoRA: Extreme Parameter Efficient Fine-Tuning with Vector Banks (NeurIPS 2024).
  ☆42Updated last year
• assafbk / DeciMamba
  DeciMamba: Exploring the Length Extrapolation Potential of Mamba (ICLR 2025)
  ☆31Updated 8 months ago
• badripatro / mamba360
  State Space Models
  ☆71Updated last year
• GATECH-EIC / Linearized-LLM
  [ICML 2024] When Linear Attention Meets Autoregressive Decoding: Towards More Effective and Efficient Linearized Large Language Models
  ☆35Updated last year
• htqin / IR-QLoRA
  [ICML 2024 Oral] This project is the official implementation of our Accurate LoRA-Finetuning Quantization of LLMs via Information Retenti…
  ☆67Updated last year
• THUDM / Awesome-Parameter-Efficient-Fine-Tuning-for-Foundation-Models
  Parameter-Efficient Fine-Tuning for Foundation Models
  ☆102Updated 8 months ago
• Adamdad / rational_kat_cu
  ☆76Updated 10 months ago
• yikangshen / MoA
  Mixture of Attention Heads
  ☆51Updated 3 years ago
• AwesomeSeq / Comba-triton
  ☆49Updated 5 months ago
• Itamarzimm / UnifiedImplicitAttnRepr
  [ICLR 2025] Official Code Release for Explaining Modern Gated-Linear RNNs via a Unified Implicit Attention Formulation
  ☆47Updated 9 months ago
• BICLab / MetaLA
  Offical implementation of "MetaLA: Unified Optimal Linear Approximation to Softmax Attention Map" (NeurIPS2024 Oral)
  ☆32Updated 10 months ago
• JieShibo / MoLE
  [ICML 2025 Oral] Mixture of Lookup Experts
  ☆56Updated last week
• fla-org / flash-bidirectional-linear-attention
  Triton implement of bi-directional (non-causal) linear attention
  ☆57Updated 10 months ago
• kyegomez / Mixture-of-Depths
  Implementation of the paper: "Mixture-of-Depths: Dynamically allocating compute in transformer-based language models"
  ☆111Updated last week
• Chaos96 / fourierft
  ☆151Updated last year
• annosubmission / GRC-Cache
  ☆16Updated 2 years ago
• chuanyang-Zheng / DAPE
  The this is the official implementation of "DAPE: Data-Adaptive Positional Encoding for Length Extrapolation"
  ☆39Updated last year
• MambaMixer / M2
  ☆48Updated last year
• OpenNLPLab / HGRN2
  HGRN2: Gated Linear RNNs with State Expansion
  ☆55Updated last year
• csarron / PuMer
  [ACL 2023] PuMer: Pruning and Merging Tokens for Efficient Vision Language Models
  ☆36Updated last year
• Caiyun-AI / MUDDFormer
  ☆85Updated 7 months ago
• pprp / Pruner-Zero
  [ICML24] Pruner-Zero: Evolving Symbolic Pruning Metric from scratch for LLMs
  ☆98Updated last year
• pengzhangzhi / Awesome-Mamba
  Awesome list of papers that extend Mamba to various applications.
  ☆139Updated 6 months ago
• scale-lab / MTLoRA
  The official implementation for MTLoRA: A Low-Rank Adaptation Approach for Efficient Multi-Task Learning (CVPR '24)
  ☆69Updated 5 months ago
• TsinghuaC3I / Fourier-Position-Embedding
  [ICML 2025] Fourier Position Embedding: Enhancing Attention’s Periodic Extension for Length Generalization
  ☆104Updated 6 months ago
• TsinghuaC3I / SoRA
  [EMNLP 2023, Main Conference] Sparse Low-rank Adaptation of Pre-trained Language Models
  ☆85Updated last year
• zhixuan-lin / forgetting-transformer
  [ICLR 2025 & COLM 2025] Official PyTorch implementation of the Forgetting Transformer and Adaptive Computation Pruning
  ☆134Updated last month
• ziplab / QLLM
  [ICLR 2024] This is the official PyTorch implementation of "QLLM: Accurate and Efficient Low-Bitwidth Quantization for Large Language Mod…
  ☆30Updated last year
• kyegomez / MoE-Mamba
  Implementation of MoE Mamba from the paper: "MoE-Mamba: Efficient Selective State Space Models with Mixture of Experts" in Pytorch and Ze…
  ☆118Updated last month