Alternatives and similar repositories for RWKV-LM

Users that are interested in RWKV-LM are comparing it to the libraries listed below

• TRI-ML / linear_open_lm
  A repository for research on medium sized language models.
  ☆76Updated last year
• BlinkDL / LinearAttentionArena
  Here we will test various linear attention designs.
  ☆59Updated last year
• RobertCsordas / moeut
  ☆79Updated 10 months ago
• frankxwang / dpo-prefix-sharing
  DPO, but faster 🚀
  ☆43Updated 6 months ago
• kyleliang919 / Online-Subspace-Descent
  This repo is based on https://github.com/jiaweizzhao/GaLore
  ☆28Updated 9 months ago
• BlinkDL / modded-nanogpt-rwkv
  RWKV-7: Surpassing GPT
  ☆91Updated 7 months ago
• SprocketLab / sparse_matrix_fine_tuning
  Official repository for ICML 2024 paper "MoRe Fine-Tuning with 10x Fewer Parameters"
  ☆20Updated last month
• NathanGodey / qfilters
  Repository for the Q-Filters method (https://arxiv.org/pdf/2503.02812)
  ☆33Updated 3 months ago
• lucasjinreal / ImageTokenizer
  imagetokenizer is a python package, helps you encoder visuals and generate visuals token ids from codebook, supports both image and video…
  ☆34Updated last year
• RWKV / ZeroCoT
  https://x.com/BlinkDL_AI/status/1884768989743882276
  ☆28Updated last month
• recursal / GoldFinch-paper
  GoldFinch and other hybrid transformer components
  ☆45Updated 11 months ago
• RobertCsordas / moe_attention
  Official repository for the paper "SwitchHead: Accelerating Transformers with Mixture-of-Experts Attention"
  ☆98Updated 8 months ago
• Qichuzyy / POA
  Official implementation of ECCV24 paper: POA
  ☆24Updated 10 months ago
• kyegomez / OpenStrawberry
  An open source replication of the stawberry method that leverages Monte Carlo Search with PPO and or DPO
  ☆29Updated 2 weeks ago
• VITA-Group / WeLore
  From GaLore to WeLore: How Low-Rank Weights Non-uniformly Emerge from Low-Rank Gradients. Ajay Jaiswal, Lu Yin, Zhenyu Zhang, Shiwei Liu,…
  ☆47Updated 2 months ago
• fal-ai-community / nano-mdm
  Tiny re-implementation of MDM in style of LLaDA and nano-gpt speedrun
  ☆52Updated 3 months ago
• fla-org / flash-bidirectional-linear-attention
  Triton implement of bi-directional (non-causal) linear attention
  ☆50Updated 4 months ago
• Infini-AI-Lab / gsm_infinite
  ☆47Updated last week
• kyleliang919 / Super_Muon
  ☆56Updated 3 months ago
• TsinghuaC3I / Fourier-Position-Embedding
  [ICML 2025] Fourier Position Embedding: Enhancing Attention’s Periodic Extension for Length Generalization
  ☆71Updated 3 weeks ago
• yynil / RWKVInside
  ☆36Updated last month
• pixeli99 / MixLN
  [ICLR 2025] Official Pytorch Implementation of "Mix-LN: Unleashing the Power of Deeper Layers by Combining Pre-LN and Post-LN" by Pengxia…
  ☆24Updated 6 months ago
• g-luo / vlm_cross_modal_reps
  Official PyTorch Implementation for Vision-Language Models Create Cross-Modal Task Representations, ICML 2025
  ☆27Updated last month
• AnonymousAlethiometer / SGD_SaI
  Official PyTorch Implementation for Paper "No More Adam: Learning Rate Scaling at Initialization is All You Need"
  ☆52Updated 4 months ago
• itsnamgyu / block-transformer
  Block Transformer: Global-to-Local Language Modeling for Fast Inference (NeurIPS 2024)
  ☆158Updated 2 months ago
• OpenSparseLLMs / Linearization
  ☆51Updated 3 months ago
• scitix / MEAP
  Mask-Enhanced Autoregressive Prediction: Pay Less Attention to Learn More
  ☆31Updated last month
• zhixuan-lin / forgetting-transformer
  [ICLR 2025] Official PyTorch implementation of "Forgetting Transformer: Softmax Attention with a Forget Gate"
  ☆108Updated last month
• LiqunMa / FBI-LLM
  FBI-LLM: Scaling Up Fully Binarized LLMs from Scratch via Autoregressive Distillation
  ☆49Updated 11 months ago
• IST-DASLab / QuEST
  Work in progress.
  ☆69Updated 2 weeks ago