yuunnn-w / RWKV_Pytorch
This is an inference framework for the RWKV large language model implemented purely in native PyTorch. The official native implementation is overly complex and lacks extensibility. Let's join the flexible PyTorch ecosystem and open-source it together!
☆118Updated 3 months ago
Related projects ⓘ
Alternatives and complementary repositories for RWKV_Pytorch
- ☆82Updated this week
- VisualRWKV is the visual-enhanced version of the RWKV language model, enabling RWKV to handle various visual tasks.☆180Updated this week
- ☆10Updated last year
- This project is established for real-time training of the RWKV model.☆50Updated 5 months ago
- rwkv finetuning☆35Updated 6 months ago
- ☆12Updated 2 months ago
- RWKV infctx trainer, for training arbitary context sizes, to 10k and beyond!☆133Updated 2 months ago
- RAG SYSTEM FOR RWKV☆35Updated this week
- Evaluating LLMs with Dynamic Data☆68Updated this week
- ☆14Updated this week
- 用户友好、开箱即用的 RWKV Prompts 示例，适用于所有用户。Awesome RWKV Prompts for general users, more user-friendly, ready-to-use prompt examples.☆29Updated 3 months ago
- Get down and dirty with FlashAttention2.0 in pytorch, plug in and play no complex CUDA kernels☆95Updated last year
- continous batching and parallel acceleration for RWKV6☆23Updated 4 months ago
- ☆81Updated 5 months ago
- 更简单的微调，提供便捷脚本，微调说明☆30Updated 3 weeks ago
- RWKV in nanoGPT style☆176Updated 5 months ago
- State tuning tunes the state☆26Updated 7 months ago
- Lightning Attention-2: A Free Lunch for Handling Unlimited Sequence Lengths in Large Language Models☆184Updated 6 months ago
- ☆14Updated 2 weeks ago
- ☆44Updated 7 months ago
- [EMNLP 2024] RWKV-CLIP: A Robust Vision-Language Representation Learner☆110Updated this week
- 实现Blip2RWKV+QFormer的多模态图文对话大模型，使用Two-Step Cognitive Psychology Prompt方法，仅3B参数的模型便能够出现类人因果思维链。对标MiniGPT-4，ImageBind等图文对话大语言模型，力求以更小的算力和资源实…☆36Updated last year
- The CUDA version of the RWKV language model ( https://github.com/BlinkDL/RWKV-LM )☆212Updated 5 months ago
- This is a personal reimplementation of Google's Infini-transformer, utilizing a small 2b model. The project includes both model and train…☆52Updated 6 months ago
- SUS-Chat: Instruction tuning done right☆47Updated 9 months ago
- Efficient implementations of state-of-the-art linear attention models in Pytorch and Triton☆13Updated 2 weeks ago
- Skywork-MoE: A Deep Dive into Training Techniques for Mixture-of-Experts Language Models☆127Updated 4 months ago
- A QQ Chatbot based on RWKV (W.I.P.)☆78Updated 11 months ago
- Lion and Adam optimization comparison☆56Updated last year
- Mixture-of-Experts (MoE) Language Model☆180Updated 2 months ago