JL-er / WorldRWKVLinks
The WorldRWKV project aims to implement training and inference across various modalities using the RWKV7 architecture. By leveraging different encoders, the project allows for flexible modality switching and aspires to achieve end-to-end cross-modal inference.
☆53Updated 2 weeks ago
Alternatives and similar repositories for WorldRWKV
Users that are interested in WorldRWKV are comparing it to the libraries listed below
Sorting:
- ☆18Updated 7 months ago
- ☆139Updated last month
- Reinforcement Learning Toolkit for RWKV.(v6,v7,ARWKV) Distillation,SFT,RLHF(DPO,ORPO), infinite context training, Aligning. Exploring the…☆48Updated last month
- RWKV-LM-V7(https://github.com/BlinkDL/RWKV-LM) Under Lightning Framework☆42Updated 2 weeks ago
- ☆38Updated 3 months ago
- Efficient implementations of state-of-the-art linear attention models in Pytorch and Triton☆41Updated last week
- State tuning tunes the state☆35Updated 5 months ago
- VisualRWKV is the visual-enhanced version of the RWKV language model, enabling RWKV to handle various visual tasks.☆233Updated 2 months ago
- ☆22Updated 7 months ago
- [EMNLP 2024] RWKV-CLIP: A Robust Vision-Language Representation Learner☆140Updated 2 months ago
- A large-scale RWKV v6, v7(World, PRWKV, Hybrid-RWKV) inference. Capable of inference by combining multiple states(Pseudo MoE). Easy to de…☆40Updated last week
- ☆34Updated last year
- RWKV-X is a Linear Complexity Hybrid Language Model based on the RWKV architecture, integrating Sparse Attention to improve the model's l…☆46Updated 3 weeks ago
- A specialized RWKV-7 model for Othello(a.k.a. Reversi) that predicts legal moves, evaluates positions, and performs in-context search. It…☆41Updated 6 months ago
- MiSS is a novel PEFT method that features a low-rank structure but introduces a new update mechanism distinct from LoRA, achieving an exc…☆20Updated 3 weeks ago
- This is an inference framework for the RWKV large language model implemented purely in native PyTorch. The official native implementation…☆132Updated last year
- https://x.com/BlinkDL_AI/status/1884768989743882276☆28Updated 3 months ago
- RAG SYSTEM FOR RWKV☆50Updated 8 months ago
- RWKV infctx trainer, for training arbitary context sizes, to 10k and beyond!☆148Updated 11 months ago
- [ICML'24 Oral] The official code of "DiJiang: Efficient Large Language Models through Compact Kernelization", a novel DCT-based linear at…☆102Updated last year
- RWKV-7: Surpassing GPT☆94Updated 8 months ago
- Evaluating LLMs with Dynamic Data☆91Updated 2 weeks ago
- RWKV is an RNN with transformer-level LLM performance. It can be directly trained like a GPT (parallelizable). So it's combining the best…☆51Updated 4 months ago
- This project is to extend RWKV LM's capabilities including sequence classification/embedding/peft/cross encoder/bi encoder/multi modaliti…☆10Updated last year
- This repo is an exploratory experiment to enable frozen pretrained RWKV language models to accept speech modality input. We followed the …☆53Updated 7 months ago
- rwkv finetuning☆37Updated last year
- Pytorch implementation of https://arxiv.org/html/2404.07143v1☆21Updated last year
- imagetokenizer is a python package, helps you encoder visuals and generate visuals token ids from codebook, supports both image and video…☆35Updated last year
- RWKV in nanoGPT style☆191Updated last year
- The official implement of Freeze-Omni.☆13Updated 3 weeks ago