pytorch / torchtitan
A native PyTorch Library for large model training
☆2,623Updated this week
Related projects ⓘ
Alternatives and complementary repositories for torchtitan
- PyTorch native quantization and sparsity for training and inference☆1,585Updated this week
- Make PyTorch models up to 40% faster! Thunder is a source to source compiler for PyTorch. It enables using different hardware executors a…☆1,199Updated this week
- Efficient Triton Kernels for LLM Training☆3,454Updated this week
- PyTorch native finetuning library☆4,336Updated this week
- Tile primitives for speedy kernels☆1,658Updated this week
- Training LLMs with QLoRA + FSDP☆1,418Updated last week
- A simple, performant and scalable Jax LLM!☆1,532Updated this week
- Minimalistic large language model 3D-parallelism training☆1,260Updated this week
- NanoGPT (124M) quality in 7.8 8xH100-minutes☆1,033Updated this week
- Puzzles for learning Triton☆1,135Updated this week
- nanoGPT style version of Llama 3.1☆1,246Updated 3 months ago
- Schedule-Free Optimization in PyTorch☆1,898Updated 2 weeks ago
- Freeing data processing from scripting madness by providing a set of platform-agnostic customizable pipeline processing blocks.☆2,045Updated this week
- Simple and efficient pytorch-native transformer text generation in <1000 LOC of python.☆5,669Updated last month
- Reaching LLaMA2 Performance with 0.1M Dollars☆960Updated 3 months ago
- GaLore: Memory-Efficient LLM Training by Gradient Low-Rank Projection☆1,435Updated 3 weeks ago
- Tools for merging pretrained large language models.☆4,816Updated 2 weeks ago
- ☆892Updated last month
- UNet diffusion model in pure CUDA☆584Updated 4 months ago
- Implementation of the training framework proposed in Self-Rewarding Language Model, from MetaAI☆1,336Updated 7 months ago
- Official Pytorch repository for Extreme Compression of Large Language Models via Additive Quantization https://arxiv.org/pdf/2401.06118.p…☆1,170Updated last week
- SGLang is a fast serving framework for large language models and vision language models.☆6,127Updated this week
- Repository for Meta Chameleon, a mixed-modal early-fusion foundation model from FAIR.☆1,840Updated 3 months ago
- Official implementation of "Samba: Simple Hybrid State Space Models for Efficient Unlimited Context Language Modeling"☆803Updated 3 months ago
- Efficient implementations of state-of-the-art linear attention models in Pytorch and Triton☆1,339Updated this week
- ReFT: Representation Finetuning for Language Models☆1,159Updated 2 weeks ago
- Modeling, training, eval, and inference code for OLMo☆4,645Updated this week
- An Extensible Deep Learning Library☆1,874Updated this week
- Deep learning for dummies. All the practical details and useful utilities that go into working with real models.☆715Updated last month