HazyResearch / nanoGPT-TK
The simplest, fastest repository for training/finetuning medium-sized GPTs. Now, with kittens!
☆45Updated last month
Related projects: ⓘ
- ☆29Updated 2 weeks ago
- Simple and efficient pytorch-native transformer training and inference (batched)☆53Updated 5 months ago
- Experiment of using Tangent to autodiff triton☆66Updated 7 months ago
- Cold Compress is a hackable, lightweight, and open-source toolkit for creating and benchmarking cache compression methods built on top of…☆73Updated last month
- Experiments for efforts to train a new and improved t5☆76Updated 5 months ago
- ☆55Updated 9 months ago
- A MAD laboratory to improve AI architecture designs 🧪☆84Updated 4 months ago
- [ICML 24 NGSM workshop] Associative Recurrent Memory Transformer implementation and scripts for training and evaluating☆26Updated last week
- Tree Attention: Topology-aware Decoding for Long-Context Attention on GPU clusters☆94Updated 2 weeks ago
- ☆53Updated 8 months ago
- ☆68Updated 2 months ago
- ☆50Updated last month
- A place to store reusable transformer components of my own creation or found on the interwebs☆43Updated 3 weeks ago
- Collection of autoregressive model implementation☆62Updated 2 weeks ago
- ☆66Updated 3 months ago
- ☆20Updated last year
- ☆18Updated last month
- Transformer with Mu-Parameterization, implemented in Jax/Flax. Supports FSDP on TPU pods.☆29Updated 3 weeks ago
- ☆48Updated 3 months ago
- ☆40Updated 4 months ago
- Simple and fast low-bit matmul kernels in CUDA☆48Updated this week
- Make triton easier☆39Updated 3 months ago
- GoldFinch and other hybrid transformer components☆38Updated 2 months ago
- Triton Implementation of HyperAttention Algorithm☆46Updated 9 months ago
- ☆29Updated 3 weeks ago
- ☆27Updated this week
- some common Huggingface transformers in maximal update parametrization (µP)☆76Updated 2 years ago
- ☆38Updated 8 months ago
- The source code of our work "Prepacking: A Simple Method for Fast Prefilling and Increased Throughput in Large Language Models"☆57Updated 5 months ago
- Engineering the state of RNN language models (Mamba, RWKV, etc.)☆31Updated 3 months ago