BlinkDL / modded-nanogpt-rwkv
RWKV-7: Surpassing GPT
☆44Updated this week
Related projects ⓘ
Alternatives and complementary repositories for modded-nanogpt-rwkv
- ☆49Updated 8 months ago
- GoldFinch and other hybrid transformer components☆39Updated 4 months ago
- An easy-to-understand framework for LLM samplers that rewind and revise generated tokens☆113Updated 3 weeks ago
- Demonstration that finetuning RoPE model on larger sequences than the pre-trained model adapts the model context limit☆63Updated last year
- A byte-level decoder architecture that matches the performance of tokenized Transformers.☆59Updated 6 months ago
- The simplest, fastest repository for training/finetuning medium-sized GPTs.☆84Updated last week
- ☆40Updated this week
- ☆43Updated 4 months ago
- Experiments for efforts to train a new and improved t5☆76Updated 7 months ago
- Muon optimizer for neural networks: >30% extra sample efficiency, <3% wallclock overhead☆109Updated last week
- Tree Attention: Topology-aware Decoding for Long-Context Attention on GPU clusters☆104Updated last month
- [WIP] Transformer to embed Danbooru labelsets☆13Updated 7 months ago
- ☆118Updated 3 months ago
- QLoRA with Enhanced Multi GPU Support☆36Updated last year
- Collection of autoregressive model implementation☆67Updated this week
- Fast modular code to create and train cutting edge LLMs☆65Updated 6 months ago
- Zeus LLM Trainer is a rewrite of Stanford Alpaca aiming to be the trainer for all Large Language Models☆69Updated last year
- Lightweight tools for quick and easy LLM demo's☆26Updated last month
- ☆43Updated 2 months ago
- an implementation of Self-Extend, to expand the context window via grouped attention☆118Updated 10 months ago
- ☆62Updated 3 months ago
- Normalized Transformer (nGPT)☆66Updated this week
- ☆73Updated 4 months ago
- ☆20Updated last year
- ☆27Updated 5 months ago
- QuIP quantization☆46Updated 8 months ago
- ☆33Updated 6 months ago
- Latent Large Language Models☆16Updated 2 months ago
- Full finetuning of large language models without large memory requirements☆93Updated 10 months ago