Blealtan / RWKV-LM-LoRA
RWKV is a RNN with transformer-level LLM performance. It can be directly trained like a GPT (parallelizable). So it's combining the best of RNN and transformer - great performance, fast inference, saves VRAM, fast training, "infinite" ctx_len, and free sentence embedding.
☆407Updated last year
Related projects ⓘ
Alternatives and complementary repositories for RWKV-LM-LoRA
- ☆81Updated 6 months ago
- A torchless, c++ rwkv implementation using 8bit quantization, written in cuda/hip/vulkan for maximum compatibility and minimum dependenci…☆307Updated 9 months ago
- This project is established for real-time training of the RWKV model.☆50Updated 6 months ago
- RWKV infctx trainer, for training arbitary context sizes, to 10k and beyond!☆133Updated 3 months ago
- 📖 — Notebooks related to RWKV☆59Updated last year
- ☆84Updated last week
- ☆534Updated 11 months ago
- A QQ Chatbot based on RWKV (W.I.P.)☆78Updated last year
- A high-throughput and memory-efficient inference and serving engine for LLMs☆130Updated 4 months ago
- ChatGPT-like Web UI for RWKVstic☆100Updated last year
- ☆454Updated last year
- Train llama with lora on one 4090 and merge weight of lora to work as stanford alpaca.☆50Updated last year
- INT4/INT5/INT8 and FP16 inference on CPU for RWKV language model☆1,424Updated 3 months ago
- Official repository for LongChat and LongEval☆512Updated 5 months ago
- Framework agnostic python runtime for RWKV models☆145Updated last year
- Enhancing LangChain prompts to work better with RWKV models☆34Updated last year
- ☆527Updated 10 months ago
- 使用Gradio制作的基于RWKV的角色扮演的webui☆229Updated last week
- Simple, hackable and fast implementation for training/finetuning medium-sized LLaMA-based models☆153Updated this week
- 4 bits quantization of LLaMa using GPTQ☆130Updated last year
- Fast Inference Solutions for BLOOM☆560Updated last month
- Script and instruction how to fine-tune large RWKV model on your data for Alpaca dataset☆31Updated last year
- The all-in-one RWKV runtime box with embed, RAG, AI agents, and more.☆480Updated last week
- Multi-language Enhanced LLaMA☆301Updated last year
- Tune any FALCON in 4-bit☆468Updated last year
- rwkv_chatbot☆62Updated last year
- Plain pytorch implementation of LLaMA☆189Updated last year
- Implementation of the RWKV language model in pure WebGPU/Rust.☆240Updated last week
- Finetuning Large Language Models on One Consumer GPU in 2 Bits☆707Updated 5 months ago
- A japanese finetuned instruction LLaMA☆126Updated last year