yuunnn-w / RWKV_Pytorch
This is an inference framework for the RWKV large language model implemented purely in native PyTorch. The official native implementation is overly complex and lacks extensibility. Let's join the flexible PyTorch ecosystem and open-source it together!
☆126Updated 8 months ago
Alternatives and similar repositories for RWKV_Pytorch:
Users that are interested in RWKV_Pytorch are comparing it to the libraries listed below
- ☆112Updated last week
- This project is established for real-time training of the RWKV model.☆49Updated 10 months ago
- ☆18Updated 2 months ago
- RAG SYSTEM FOR RWKV☆45Updated 3 months ago
- ☆13Updated 3 months ago
- Evaluating LLMs with Dynamic Data☆78Updated last month
- VisualRWKV is the visual-enhanced version of the RWKV language model, enabling RWKV to handle various visual tasks.☆212Updated 2 weeks ago
- RWKV infctx trainer, for training arbitary context sizes, to 10k and beyond!☆148Updated 7 months ago
- rwkv finetuning☆36Updated 11 months ago
- ☆82Updated 10 months ago
- Efficient implementations of state-of-the-art linear attention models in Pytorch and Triton☆20Updated this week
- ☆22Updated 2 months ago
- ☆10Updated last year
- ☆124Updated last year
- Lightning Attention-2: A Free Lunch for Handling Unlimited Sequence Lengths in Large Language Models☆272Updated last month
- continous batching and parallel acceleration for RWKV6☆24Updated 8 months ago
- ☆38Updated last week
- Direct Preference Optimization for RWKV, aiming for RWKV-5 and 6.☆11Updated last year
- Inference RWKV v5, v6 and v7 with Qualcomm AI Engine Direct SDK☆59Updated this week
- 实现Blip2RWKV+QFormer的多模态图文对话大模型,使用Two-Step Cognitive Psychology Prompt方法,仅3B参数的模型便能够出现类人因果思维链。对标MiniGPT-4,ImageBind等图文对话大语言模型,力求以更小的算力和资源实…☆38Updated last year
- Reinforcement Learning Toolkit for RWKV.(v6,v7,ARWKV) Distillation,SFT,RLHF(DPO,ORPO), infinite context training, Aligning. Exploring the…☆35Updated this week
- State tuning tunes the state☆31Updated last month
- RWKV, in easy to read code☆71Updated 3 months ago
- RWKV in nanoGPT style☆187Updated 9 months ago
- The CUDA version of the RWKV language model ( https://github.com/BlinkDL/RWKV-LM )☆220Updated 3 months ago
- Low-bit optimizers for PyTorch☆125Updated last year
- 更简单的微调,提供便捷脚本,微调说明☆33Updated last month
- Implementation of the RWKV language model in pure WebGPU/Rust.☆295Updated last week
- Fast modular code to create and train cutting edge LLMs☆66Updated 10 months ago
- A quantization algorithm for LLM☆136Updated 9 months ago