Alternatives and similar repositories for TorchRWKV

Users that are interested in TorchRWKV are comparing it to the libraries listed below

• lucidrains / light-recurrent-unit-pytorch
  Implementation of a Light Recurrent Unit in Pytorch
  ☆49Updated last year
• lucidrains / transformer-lm-gan
  Explorations into adversarial losses on top of autoregressive loss for language modeling
  ☆41Updated 3 weeks ago
• lucidrains / simplicial-attention
  Implementation of 2-simplicial attention proposed by Clift et al. (2019) and the recent attempt to make practical in Fast and Simplex, Ro…
  ☆46Updated 4 months ago
• OpenMOSE / RWKV-Infer
  A large-scale RWKV v7(World, PRWKV, Hybrid-RWKV) inference. Capable of inference by combining multiple states(Pseudo MoE). Easy to deploy…
  ☆46Updated 2 months ago
• BlinkDL / LinearAttentionArena
  Here we will test various linear attention designs.
  ☆62Updated last year
• nanowell / Q-Sparse-LLM
  My Implementation of Q-Sparse: All Large Language Models can be Fully Sparsely-Activated
  ☆33Updated last year
• sustcsonglin / gated_linear_attention_layer
  ☆32Updated 2 years ago
• johanwind / wind_rwkv
  ☆27Updated 5 months ago
• RWKV / ZeroCoT
  https://x.com/BlinkDL_AI/status/1884768989743882276
  ☆28Updated 8 months ago
• kazuki-irie / kv-memory-brain
  Official Code Repository for the paper "Key-value memory in the brain"
  ☆31Updated 10 months ago
• BBuf / flash-rwkv
  ☆32Updated last year
• Doraemonzzz / hgru2-pytorch
  ☆24Updated last year
• proger / hippogriff
  Griffin MQA + Hawk Linear RNN Hybrid
  ☆89Updated last year
• lucidrains / strassen-attention
  Implementation of Strassen attention, from Kozachinskiy et al. of National Center of AI in Chile
  ☆41Updated 6 months ago
• recursal / GoldFinch-paper
  GoldFinch and other hybrid transformer components
  ☆45Updated last year
• acosharma / elita-transformer
  Official Repository for Efficient Linear-Time Attention Transformers.
  ☆18Updated last year
• lukasVierling / FaceRWKV
  Course Project for COMP4471 on RWKV
  ☆17Updated last year
• howard-hou / RWKV-X
  RWKV-X is a Linear Complexity Hybrid Language Model based on the RWKV architecture, integrating Sparse Attention to improve the model's l…
  ☆53Updated 5 months ago
• Jellyfish042 / LongMamba
  Some preliminary explorations of Mamba's context scaling.
  ☆13Updated last year
• BlinkDL / WorldModel
  Let us make Psychohistory (as in Asimov) a reality, and accessible to everyone. Useful for LLM grounding and games / fiction / business /…
  ☆40Updated 2 years ago
• Doraemonzzz / hgru-pytorch
  ☆29Updated last year
• OpenNLPLab / HGRN2
  HGRN2: Gated Linear RNNs with State Expansion
  ☆56Updated last year
• kyegomez / Blockwise-Parallel-Transformer
  32 times longer context window than vanilla Transformers and up to 4 times longer than memory efficient Transformers.
  ☆49Updated 2 years ago
• SmerkyG / RWKV_Explained
  RWKV, in easy to read code
  ☆72Updated 9 months ago
• Triang-jyed-driung / rwkv7mini
  RWKV-7 mini
  ☆11Updated 9 months ago
• RWKV-Vibe / rwkv-fla
  Efficient implementations of state-of-the-art linear attention models in Pytorch and Triton
  ☆46Updated 4 months ago
• lucidrains / self-reasoning-tokens-pytorch
  Exploration into the proposed "Self Reasoning Tokens" by Felipe Bonetto
  ☆57Updated last year
• Jellyfish042 / RWKV-StateTuning
  State tuning tunes the state
  ☆35Updated 10 months ago
• glassroom / heinsen_attention
  Reference implementation of "Softmax Attention with Constant Cost per Token" (Heinsen, 2024)
  ☆24Updated last year
• BlinkDL / modded-nanogpt-rwkv
  RWKV-7: Surpassing GPT
  ☆103Updated last year