yuunnn-w / RWKV_Pytorch
This is an inference framework for the RWKV large language model implemented purely in native PyTorch. The official native implementation is overly complex and lacks extensibility. Let's join the flexible PyTorch ecosystem and open-source it together!
☆125Updated 7 months ago
Alternatives and similar repositories for RWKV_Pytorch:
Users that are interested in RWKV_Pytorch are comparing it to the libraries listed below
- ☆108Updated this week
- ☆18Updated 2 months ago
- VisualRWKV is the visual-enhanced version of the RWKV language model, enabling RWKV to handle various visual tasks.☆211Updated last week
- RAG SYSTEM FOR RWKV☆45Updated 3 months ago
- ☆13Updated 2 months ago
- ☆21Updated 2 months ago
- This project is established for real-time training of the RWKV model.☆49Updated 9 months ago
- 用户友好、开箱即用的 RWKV Prompts 示例，适用于所有用户。Awesome RWKV Prompts for general users, more user-friendly, ready-to-use prompt examples.☆33Updated last month
- rwkv finetuning☆36Updated 10 months ago
- Reinforcement Learning Toolkit for RWKV.(v6,v7,ARWKV) Distillation,SFT,RLHF(DPO,ORPO), infinite context training, Aligning. Exploring the…☆33Updated last week
- RWKV infctx trainer, for training arbitary context sizes, to 10k and beyond!☆148Updated 7 months ago
- ☆10Updated last year
- Evaluating LLMs with Dynamic Data☆77Updated last month
- ☆124Updated last year
- continous batching and parallel acceleration for RWKV6☆24Updated 8 months ago
- Efficient implementations of state-of-the-art linear attention models in Pytorch and Triton☆19Updated this week
- This repo is an exploratory experiment to enable frozen pretrained RWKV language models to accept speech modality input. We followed the …☆47Updated 2 months ago
- A large-scale RWKV v6, v7(World, ARWKV) inference. Capable of inference by combining multiple states(Pseudo MoE). Easy to deploy on docke…☆31Updated 2 weeks ago
- RWKV, in easy to read code☆69Updated 3 months ago
- State tuning tunes the state☆30Updated last month
- Get down and dirty with FlashAttention2.0 in pytorch, plug in and play no complex CUDA kernels☆102Updated last year
- Inference RWKV v5, v6 and v7 with Qualcomm AI Engine Direct SDK☆55Updated this week
- ☆81Updated 9 months ago
- RWKV in nanoGPT style☆187Updated 9 months ago
- Lightning Attention-2: A Free Lunch for Handling Unlimited Sequence Lengths in Large Language Models☆269Updated 2 weeks ago
- Triton Documentation in Chinese Simplified / Triton 中文文档☆58Updated 2 months ago
- Implementation of the RWKV language model in pure WebGPU/Rust.☆294Updated this week
- Direct Preference Optimization for RWKV, aiming for RWKV-5 and 6.☆11Updated last year
- Fast modular code to create and train cutting edge LLMs☆66Updated 9 months ago