yuunnn-w / RWKV_Pytorch
This is an inference framework for the RWKV large language model implemented purely in native PyTorch. The official native implementation is overly complex and lacks extensibility. Let's join the flexible PyTorch ecosystem and open-source it together!
☆128Updated 9 months ago
Alternatives and similar repositories for RWKV_Pytorch
Users that are interested in RWKV_Pytorch are comparing it to the libraries listed below
Sorting:
- ☆121Updated 3 weeks ago
- RAG SYSTEM FOR RWKV☆46Updated 5 months ago
- VisualRWKV is the visual-enhanced version of the RWKV language model, enabling RWKV to handle various visual tasks.☆222Updated last month
- rwkv finetuning☆36Updated last year
- ☆18Updated 4 months ago
- ☆13Updated 4 months ago
- ☆22Updated 4 months ago
- 用户友好、开箱即用的 RWKV Prompts 示例,适用于所有用户。Awesome RWKV Prompts for general users, more user-friendly, ready-to-use prompt examples.☆34Updated 3 months ago
- Efficient implementations of state-of-the-art linear attention models in Pytorch and Triton☆28Updated last week
- The WorldRWKV project aims to implement training and inference across various modalities using the RWKV7 architecture. By leveraging diff…☆44Updated last month
- This project is established for real-time training of the RWKV model.☆49Updated 11 months ago
- Reinforcement Learning Toolkit for RWKV.(v6,v7,ARWKV) Distillation,SFT,RLHF(DPO,ORPO), infinite context training, Aligning. Exploring the…☆40Updated this week
- Evaluating LLMs with Dynamic Data☆86Updated 2 weeks ago
- RWKV infctx trainer, for training arbitary context sizes, to 10k and beyond!☆148Updated 8 months ago
- ☆124Updated last year
- State tuning tunes the state☆32Updated 3 months ago
- continous batching and parallel acceleration for RWKV6☆24Updated 10 months ago
- Lightning Attention-2: A Free Lunch for Handling Unlimited Sequence Lengths in Large Language Models☆284Updated 2 months ago
- This repo is an exploratory experiment to enable frozen pretrained RWKV language models to accept speech modality input. We followed the …☆49Updated 4 months ago
- Inference RWKV with multiple supported backends.☆43Updated this week
- Inference RWKV v5, v6 and v7 with Qualcomm AI Engine Direct SDK☆64Updated this week
- Get down and dirty with FlashAttention2.0 in pytorch, plug in and play no complex CUDA kernels☆104Updated last year
- ☆82Updated 11 months ago
- Direct Preference Optimization for RWKV, aiming for RWKV-5 and 6.☆11Updated last year
- Triton Documentation in Chinese Simplified / Triton 中文文档☆69Updated 3 weeks ago
- RWKV in nanoGPT style☆189Updated 11 months ago
- ☆10Updated last year
- A specialized RWKV-7 model for Othello(a.k.a. Reversi) that predicts legal moves, evaluates positions, and performs in-context search. It…☆40Updated 3 months ago
- 实现Blip2RWKV+QFormer的多模态图文对话大模型,使用Two-Step Cognitive Psychology Prompt方法,仅3B参数的模型便能够出现类人因果思维链。对标MiniGPT-4,ImageBind等图文对话大语言模型,力求以更小的算力和资源实…☆38Updated last year
- RWKV, in easy to read code☆72Updated last month