Alternatives and similar repositories for RWKV-block

Users that are interested in RWKV-block are comparing it to the libraries listed below

• lucidrains / hyper-connections
  Attempt to make multiple residual streams from Bytedance's Hyper-Connections paper accessible to the public
  ☆172Updated last week
• RWKV / ZeroCoT
  https://x.com/BlinkDL_AI/status/1884768989743882276
  ☆28Updated 9 months ago
• howard-hou / RWKV-X
  RWKV-X is a Linear Complexity Hybrid Language Model based on the RWKV architecture, integrating Sparse Attention to improve the model's l…
  ☆54Updated last month
• BlinkDL / modded-nanogpt-rwkv
  RWKV-7: Surpassing GPT
  ☆104Updated last year
• bluorion-com / ZClip
  Official implementation of the paper: "ZClip: Adaptive Spike Mitigation for LLM Pre-Training".
  ☆144Updated 2 months ago
• SmerkyG / gptcore
  Fast modular code to create and train cutting edge LLMs
  ☆68Updated last year
• lucidrains / nGPT-pytorch
  Quick implementation of nGPT, learning entirely on the hypersphere, from NvidiaAI
  ☆293Updated 8 months ago
• fal-ai-community / NativeSparseAttention
  research impl of Native Sparse Attention (2502.11089)
  ☆63Updated 11 months ago
• dvruette / gidd
  Code accompanying the paper "Generalized Interpolating Discrete Diffusion"
  ☆113Updated 8 months ago
• KindXiaoming / grow-crystals
  Getting crystal-like representations with harmonic loss
  ☆195Updated 10 months ago
• NX-AI / mlstm_kernels
  Tiled Flash Linear Attention library for fast and efficient mLSTM Kernels.
  ☆84Updated 2 months ago
• yynil / RWKVInside
  ☆41Updated 9 months ago
• ethansmith2000 / TransformerExperiments
  ☆19Updated 2 months ago
• main-horse / hnet-old
  H-Net Dynamic Hierarchical Architecture
  ☆81Updated 5 months ago
• proger / hippogriff
  Griffin MQA + Hawk Linear RNN Hybrid
  ☆88Updated last year
• tanaymeh / mamba-train
  A single repo with all scripts and utils to train / fine-tune the Mamba model with or without FIM
  ☆61Updated last year
• declare-lab / EFLA
  Official Repo for Error-Free Linear Attention is a Free Lunch: Exact Solution from Continuous-Time Dynamics
  ☆71Updated 3 weeks ago
• Joluck / MiSS
  MiSS is a novel PEFT method that features a low-rank structure but introduces a new update mechanism distinct from LoRA, achieving an exc…
  ☆30Updated 2 weeks ago
• NX-AI / flashrnn
  FlashRNN - Fast RNN Kernels with I/O Awareness
  ☆174Updated 3 months ago
• recursal / GoldFinch-paper
  GoldFinch and other hybrid transformer components
  ☆45Updated last year
• zaydzuhri / softpick-attention
  Landing repository for the paper "Softpick: No Attention Sink, No Massive Activations with Rectified Softmax"
  ☆87Updated 5 months ago
• Zyphra / tree_attention
  Tree Attention: Topology-aware Decoding for Long-Context Attention on GPU clusters
  ☆131Updated last year
• lucidrains / grokfast-pytorch
  Explorations into the proposal from the paper "Grokfast, Accelerated Grokking by Amplifying Slow Gradients"
  ☆103Updated last year
• lucidrains / simplicial-attention
  Implementation of 2-simplicial attention proposed by Clift et al. (2019) and the recent attempt to make practical in Fast and Simplex, Ro…
  ☆46Updated 5 months ago
• AnonymousAlethiometer / SGD_SaI
  Official PyTorch Implementation for Paper "No More Adam: Learning Rate Scaling at Initialization is All You Need"
  ☆56Updated last year
• kjslag / spacebyte
  A byte-level decoder architecture that matches the performance of tokenized Transformers.
  ☆67Updated last year
• lucidrains / self-reasoning-tokens-pytorch
  Exploration into the proposed "Self Reasoning Tokens" by Felipe Bonetto
  ☆57Updated last year
• lucidrains / transformer-lm-gan
  Explorations into adversarial losses on top of autoregressive loss for language modeling
  ☆41Updated last month
• zhixuan-lin / forgetting-transformer
  [ICLR 2025 & COLM 2025] Official PyTorch implementation of the Forgetting Transformer and Adaptive Computation Pruning
  ☆137Updated last month
• epfml / DenseFormer
  ☆82Updated last year