Blealtan / RWKV-LM-LoRAView external linksLinks
RWKV is a RNN with transformer-level LLM performance. It can be directly trained like a GPT (parallelizable). So it's combining the best of RNN and transformer - great performance, fast inference, saves VRAM, fast training, "infinite" ctx_len, and free sentence embedding.
☆412Jul 11, 2023Updated 2 years ago
Alternatives and similar repositories for RWKV-LM-LoRA
Users that are interested in RWKV-LM-LoRA are comparing it to the libraries listed below
Sorting:
- ☆81May 15, 2024Updated last year
- RWKV infctx trainer, for training arbitary context sizes, to 10k and beyond!☆148Aug 13, 2024Updated last year
- INT4/INT5/INT8 and FP16 inference on CPU for RWKV language model☆1,562Mar 23, 2025Updated 10 months ago
- A torchless, c++ rwkv implementation using 8bit quantization, written in cuda/hip/vulkan for maximum compatibility and minimum dependenci…☆313Jan 31, 2024Updated 2 years ago
- ChatRWKV is like ChatGPT but powered by RWKV (100% RNN) language model, and open source.☆9,512Updated this week
- ☆41Mar 14, 2024Updated last year
- Script and instruction how to fine-tune large RWKV model on your data for Alpaca dataset☆31Apr 2, 2023Updated 2 years ago
- Trying to deconstruct RWKV in understandable terms☆14May 6, 2023Updated 2 years ago
- rwkv finetuning☆37Apr 22, 2024Updated last year
- RWKV (pronounced RwaKuv) is an RNN with great LLM performance, which can also be directly trained like a GPT transformer (parallelizable)…☆14,351Updated this week
- A converter and basic tester for rwkv onnx☆43Jan 29, 2024Updated 2 years ago
- This project is established for real-time training of the RWKV model.☆50May 17, 2024Updated last year
- ☆27Jul 28, 2025Updated 6 months ago
- ☆171Jan 13, 2026Updated last month
- Implementation of the RWKV language model in pure WebGPU/Rust.☆340Jan 10, 2026Updated last month
- Framework agnostic python runtime for RWKV models☆147Aug 24, 2023Updated 2 years ago
- A QQ Chatbot based on RWKV (W.I.P.)☆80Nov 16, 2023Updated 2 years ago
- The all-in-one RWKV runtime box with embed, RAG, AI agents, and more.☆601Oct 21, 2025Updated 3 months ago
- Fine-tuning RWKV-World model☆26Jun 6, 2023Updated 2 years ago
- 使用Gradio制作的基于RWKV的角色扮演的webui☆247Mar 5, 2025Updated 11 months ago
- ☆13Jun 3, 2023Updated 2 years ago
- 📖 — Notebooks related to RWKV☆58May 13, 2023Updated 2 years ago
- Implementation of "SpikeGPT: Generative Pre-trained Language Model with Spiking Neural Networks"☆878Jul 21, 2025Updated 6 months ago
- The nanoGPT-style implementation of RWKV Language Model - an RNN with GPT-level LLM performance.☆198Nov 9, 2023Updated 2 years ago
- JAX implementations of RWKV☆19Sep 26, 2023Updated 2 years ago
- BlinkDL's RWKV-v4 running in the browser☆48Mar 2, 2023Updated 2 years ago
- ChatGPT-like Web UI for RWKVstic☆100Apr 18, 2023Updated 2 years ago
- Direct Preference Optimization for RWKV, aiming for RWKV-5 and 6.☆11Mar 1, 2024Updated last year
- Run ONNX RWKV-v4 models with GPU acceleration using DirectML [Windows], or just on CPU [Windows AND Linux]; Limited to 430M model at this…☆21Mar 16, 2023Updated 2 years ago
- ☆44Mar 29, 2023Updated 2 years ago
- Enhancing LangChain prompts to work better with RWKV models☆34May 30, 2023Updated 2 years ago
- ☆74Nov 11, 2023Updated 2 years ago
- Here we will test various linear attention designs.☆62Apr 25, 2024Updated last year
- A finetuning pipeline for instruct tuning Raven 14bn using QLORA 4bit and the Ditty finetuning library☆28Jun 5, 2024Updated last year
- Easily deploy your rwkv model☆19May 5, 2023Updated 2 years ago
- The GeoV model is a large langauge model designed by Georges Harik and uses Rotary Positional Embeddings with Relative distances (RoPER).…☆121Apr 29, 2023Updated 2 years ago
- 用户友好、开箱即用的 RWKV Prompts 示例,适用于所有用户。Awesome RWKV Prompts for general users, more user-friendly, ready-to-use prompt examples.☆35Jan 24, 2025Updated last year
- A RWKV management and startup tool, full automation, only 8MB. And provides an interface compatible with the OpenAI API. RWKV is a large …☆6,215Updated this week
- Reinforcement Learning Toolkit for RWKV.(v6,v7,ARWKV) Distillation,SFT,RLHF(DPO,ORPO), infinite context training, Aligning. Exploring the…☆62Sep 19, 2025Updated 4 months ago