test-time-training / ttt-tkLinks
☆31Updated last month
Alternatives and similar repositories for ttt-tk
Users that are interested in ttt-tk are comparing it to the libraries listed below
Sorting:
- Combining SOAP and MUON☆16Updated 3 months ago
- Tiny re-implementation of MDM in style of LLaDA and nano-gpt speedrun☆52Updated 2 months ago
- Here we will test various linear attention designs.☆58Updated last year
- ☆56Updated 2 months ago
- Code for the paper "Function-Space Learning Rates"☆20Updated last month
- ☆19Updated 2 weeks ago
- Using FlexAttention to compute attention with different masking patterns☆43Updated 8 months ago
- Efficient PScan implementation in PyTorch☆16Updated last year
- supporting pytorch FSDP for optimizers☆79Updated 5 months ago
- Triton Implementation of HyperAttention Algorithm☆48Updated last year
- ☆12Updated 3 months ago
- This repo is based on https://github.com/jiaweizzhao/GaLore☆28Updated 8 months ago
- research impl of Native Sparse Attention (2502.11089)☆54Updated 3 months ago
- ☆78Updated 10 months ago
- ☆52Updated last year
- [ICLR 2025] Official PyTorch implementation of "Forgetting Transformer: Softmax Attention with a Forget Gate"☆104Updated 2 weeks ago
- Focused on fast experimentation and simplicity☆73Updated 5 months ago
- Official implementation of "The Sparse Frontier: Sparse Attention Trade-offs in Transformer LLMs"☆32Updated last month
- Stick-breaking attention☆56Updated 2 months ago
- RWKV-7: Surpassing GPT☆88Updated 6 months ago
- ☆79Updated 9 months ago
- Code accompanying the paper "Generalized Interpolating Discrete Diffusion"☆80Updated this week
- DPO, but faster 🚀☆42Updated 5 months ago
- Mixture of A Million Experts☆45Updated 10 months ago
- Custom triton kernels for training Karpathy's nanoGPT.☆19Updated 7 months ago
- ☆13Updated 2 weeks ago
- Demo of the unit_scaling library, showing how a model can be easily adapted to train in FP8.☆44Updated 10 months ago
- Parallel Associative Scan for Language Models☆18Updated last year
- Official PyTorch Implementation of the Longhorn Deep State Space Model☆50Updated 6 months ago
- Minimal but scalable implementation of large language models in JAX☆34Updated 7 months ago