yuunnn-w / RWKV_Pytorch
This is an inference framework for the RWKV large language model implemented purely in native PyTorch. The official native implementation is overly complex and lacks extensibility. Let's join the flexible PyTorch ecosystem and open-source it together!
☆128Updated 9 months ago
Alternatives and similar repositories for RWKV_Pytorch:
Users that are interested in RWKV_Pytorch are comparing it to the libraries listed below
- ☆118Updated this week
- RAG SYSTEM FOR RWKV☆45Updated 4 months ago
- VisualRWKV is the visual-enhanced version of the RWKV language model, enabling RWKV to handle various visual tasks.☆220Updated 3 weeks ago
- ☆18Updated 3 months ago
- ☆44Updated last week
- 用户友好、开箱即用的 RWKV Prompts 示例,适用于所有用户。Awesome RWKV Prompts for general users, more user-friendly, ready-to-use prompt examples.☆34Updated 2 months ago
- Efficient implementations of state-of-the-art linear attention models in Pytorch and Triton☆25Updated this week
- ☆13Updated 4 months ago
- Evaluating LLMs with Dynamic Data☆82Updated 2 months ago
- ☆22Updated 3 months ago
- RWKV infctx trainer, for training arbitary context sizes, to 10k and beyond!☆148Updated 8 months ago
- RWKV in nanoGPT style☆189Updated 10 months ago
- This project is established for real-time training of the RWKV model.☆49Updated 11 months ago
- rwkv finetuning☆36Updated last year
- ☆124Updated last year
- continous batching and parallel acceleration for RWKV6☆24Updated 9 months ago
- Lightning Attention-2: A Free Lunch for Handling Unlimited Sequence Lengths in Large Language Models☆278Updated last month
- Reinforcement Learning Toolkit for RWKV.(v6,v7,ARWKV) Distillation,SFT,RLHF(DPO,ORPO), infinite context training, Aligning. Exploring the…☆37Updated this week
- Inference RWKV v5, v6 and v7 with Qualcomm AI Engine Direct SDK☆62Updated last week
- Inference RWKV with multiple supported backends.☆40Updated this week
- This repo is an exploratory experiment to enable frozen pretrained RWKV language models to accept speech modality input. We followed the …☆48Updated 3 months ago
- ☆82Updated 11 months ago
- State tuning tunes the state☆32Updated 2 months ago
- DeepSeek Native Sparse Attention pytorch implementation☆61Updated last month
- ☆10Updated last year
- Triton Documentation in Chinese Simplified / Triton 中文文档☆66Updated last week
- 更简单的微调,提供便捷脚本,微调说明☆33Updated 2 months ago
- A specialized RWKV-7 model for Othello(a.k.a. Reversi) that predicts legal moves, evaluates positions, and performs in-context search. It…☆39Updated 2 months ago
- ☆116Updated this week
- A large-scale RWKV v6, v7(World, ARWKV, PRWKV) inference. Capable of inference by combining multiple states(Pseudo MoE). Easy to deploy o…☆35Updated 3 weeks ago