facebookresearch / spdlLinks
Scalable and Performant Data Loading
☆269Updated this week
Alternatives and similar repositories for spdl
Users that are interested in spdl are comparing it to the libraries listed below
Sorting:
- Helpful tools and examples for working with flex-attention☆811Updated this week
- PyTorch per step fault tolerance (actively under development)☆302Updated last week
- 🚀 Efficiently (pre)training foundation models with native PyTorch features, including FSDP for training and SDPA implementation of Flash…☆249Updated this week
- For optimization algorithm research and development.☆518Updated this week
- A subset of PyTorch's neural network modules, written in Python using OpenAI's Triton.☆544Updated this week
- ☆301Updated 11 months ago
- This repository contains the experimental PyTorch native float8 training UX☆223Updated 10 months ago
- PyTorch video decoding☆567Updated this week
- Implementation of 💍 Ring Attention, from Liu et al. at Berkeley AI, in Pytorch☆514Updated 2 weeks ago
- Universal Tensor Operations in Einstein-Inspired Notation for Python.☆374Updated last month
- An implementation of PSGD Kron second-order optimizer for PyTorch☆91Updated 2 months ago
- ☆286Updated last month
- Transform datasets at scale. Optimize datasets for fast AI model training.☆482Updated last week
- Efficient optimizers☆206Updated this week
- ☆188Updated 3 months ago
- Just some miscellaneous utility functions / decorators / modules related to Pytorch and Accelerate to help speed up implementation of new…☆120Updated 10 months ago
- When it comes to optimizers, it's always better to be safe than sorry☆233Updated 2 months ago
- Muon optimizer: +>30% sample efficiency with <3% wallclock overhead☆661Updated last week
- Load compute kernels from the Hub☆139Updated this week
- ☆267Updated 10 months ago
- TensorDict is a pytorch dedicated tensor container.☆925Updated this week
- Repo for "LoLCATs: On Low-Rank Linearizing of Large Language Models"☆237Updated 4 months ago
- ☆456Updated this week
- ☆150Updated 9 months ago
- ☆108Updated last year
- Minimal sharded dataset loaders, decoders, and utils for multi-modal document, image, and text datasets.☆157Updated last year
- Code and weights for the paper "Cluster and Predict Latents Patches for Improved Masked Image Modeling"☆106Updated last month
- Accelerated First Order Parallel Associative Scan☆181Updated 9 months ago
- Home for "How To Scale Your Model", a short blog-style textbook about scaling LLMs on TPUs☆380Updated last month
- The AdEMAMix Optimizer: Better, Faster, Older.☆183Updated 8 months ago