Alternatives and similar repositories for RWKV-LM:

Users that are interested in RWKV-LM are comparing it to the libraries listed below

• fla-org / flash-bidirectional-linear-attention
  Triton implement of bi-directional (non-causal) linear attention
  ☆35Updated last week
• GATECH-EIC / Linearized-LLM
  [ICML 2024] When Linear Attention Meets Autoregressive Decoding: Towards More Effective and Efficient Linearized Large Language Models
  ☆28Updated 7 months ago
• OpenNLPLab / Transnormer
  [EMNLP 2022] Official implementation of Transnormer in our EMNLP 2022 paper - The Devil in Linear Transformer
  ☆58Updated last year
• HelmholtzAI-FZJ / flex_gen
  ☆15Updated last week
• BlinkDL / LinearAttentionArena
  Here we will test various linear attention designs.
  ☆58Updated 8 months ago
• chuanyang-Zheng / DAPE
  The this is the official implementation of "DAPE: Data-Adaptive Positional Encoding for Length Extrapolation"
  ☆33Updated 3 months ago
• VITA-Group / WeLore
  From GaLore to WeLore: How Low-Rank Weights Non-uniformly Emerge from Low-Rank Gradients. Ajay Jaiswal, Lu Yin, Zhenyu Zhang, Shiwei Liu,…
  ☆42Updated 6 months ago
• BBuf / flash-rwkv
  ☆31Updated 7 months ago
• pixeli99 / MixLN
  Official Pytorch Implementation of "Mix-LN: Unleashing the Power of Deeper Layers by Combining Pre-LN and Post-LN" by Pengxiang Li, Lu Yi…
  ☆16Updated 3 weeks ago
• WailordHe / DenseSSM
  A repository for DenseSSMs
  ☆87Updated 9 months ago
• OpenNLPLab / HGRN2
  HGRN2: Gated Linear RNNs with State Expansion
  ☆52Updated 5 months ago
• htqin / IR-QLoRA
  [ICML 2024 Oral] This project is the official implementation of our Accurate LoRA-Finetuning Quantization of LLMs via Information Retenti…
  ☆60Updated 9 months ago
• berlino / gated_linear_attention
  ☆98Updated 10 months ago
• SparksJoe / Prism
  A Framework for Decoupling and Assessing the Capabilities of VLMs
  ☆40Updated 6 months ago
• kyegomez / Paper-Implementation-Template
  A simple reproducible template to implement AI research papers
  ☆22Updated 4 months ago
• BestAnHongjun / SentenceVAE
  Enable Next-sentence Prediction for Large Language Models with Faster Speed, Higher Accuracy and Longer Context
  ☆23Updated 5 months ago
• philippe-eecs / small-vision
  A big_vision inspired repo that implements a generic Auto-Encoder class capable in representation learning and generative modeling.
  ☆34Updated 6 months ago
• lucasjinreal / ImageTokenizer
  imagetokenizer is a python package, helps you encoder visuals and generate visuals token ids from codebook, supports both image and video…
  ☆30Updated 6 months ago
• g-luo / task_vectors_are_cross_modal
  Official PyTorch Implementation for Task Vectors are Cross-Modal
  ☆21Updated last month
• LiqunMa / FBI-LLM
  FBI-LLM: Scaling Up Fully Binarized LLMs from Scratch via Autoregressive Distillation
  ☆46Updated 6 months ago
• chenllliang / MMEvalPro
  Source code for MMEvalPro, a more trustworthy and efficient benchmark for evaluating LMMs
  ☆22Updated 3 months ago
• RobertCsordas / moe_attention
  Official repository for the paper "SwitchHead: Accelerating Transformers with Mixture-of-Experts Attention"
  ☆96Updated 3 months ago
• kyegomez / TTL
  Pytorch Implementation of the paper: "Learning to (Learn at Test Time): RNNs with Expressive Hidden States"
  ☆24Updated last week
• OpenNLPLab / LASP
  Linear Attention Sequence Parallelism (LASP)
  ☆74Updated 7 months ago
• Optimization-AI / fast_clip
  ☆21Updated 3 months ago
• rayleizhu / vllm-ra
  [ACL 2024] RelayAttention for Efficient Large Language Model Serving with Long System Prompts
  ☆38Updated 10 months ago
• locuslab / llava-token-compression
  ☆37Updated 2 months ago
• sail-sg / scaling-with-vocab
  [NeurIPS-2024] 📈 Scaling Laws with Vocabulary: Larger Models Deserve Larger Vocabularies https://arxiv.org/abs/2407.13623
  ☆75Updated 3 months ago
• shawntan / stickbreaking-attention
  Stick-breaking attention
  ☆41Updated last week
• Itamarzimm / UnifiedImplicitAttnRepr
  ☆36Updated 7 months ago