Lightning-AI / litDataLinks
Transform datasets at scale. Optimize datasets for fast AI model training.
☆493Updated last week
Alternatives and similar repositories for litData
Users that are interested in litData are comparing it to the libraries listed below
Sorting:
- Scalable and Performant Data Loading☆278Updated this week
- Thunder gives you PyTorch models superpowers for training and inference. Unlock out-of-the-box optimizations for performance, memory and …☆1,367Updated this week
- Helpful tools and examples for working with flex-attention☆846Updated this week
- TensorDict is a pytorch dedicated tensor container.☆929Updated this week
- PyTorch per step fault tolerance (actively under development)☆329Updated this week
- Universal Tensor Operations in Einstein-Inspired Notation for Python.☆381Updated 2 months ago
- A subset of PyTorch's neural network modules, written in Python using OpenAI's Triton.☆556Updated last week
- For optimization algorithm research and development.☆521Updated this week
- A library that contains a rich collection of performant PyTorch model metrics, a simple interface to create new metrics, a toolkit to fac…☆233Updated 5 months ago
- ☆303Updated last year
- Pipeline Parallelism for PyTorch☆768Updated 10 months ago
- Implementation of 💍 Ring Attention, from Liu et al. at Berkeley AI, in Pytorch☆521Updated last month
- Named tensors with first-class dimensions for PyTorch☆331Updated 2 years ago
- PyTorch media decoding and encoding☆589Updated this week
- Efficient optimizers☆220Updated this week
- PyTorch native quantization and sparsity for training and inference☆2,125Updated this week
- PyTorch Single Controller☆231Updated this week
- TorchFix - a linter for PyTorch-using code with autofix support☆143Updated 4 months ago
- Create powerful Hydra applications without the yaml files and boilerplate code.☆386Updated last week
- 🚀 Efficiently (pre)training foundation models with native PyTorch features, including FSDP for training and SDPA implementation of Flash…☆253Updated this week
- Best practices & guides on how to write distributed pytorch training code☆441Updated 4 months ago
- A tool to configure, launch and manage your machine learning experiments.☆162Updated this week
- Annotated version of the Mamba paper☆485Updated last year
- An implementation of PSGD Kron second-order optimizer for PyTorch☆91Updated 2 months ago
- A pytorch quantization backend for optimum☆955Updated last week
- Implementation of Diffusion Transformer (DiT) in JAX☆278Updated last year
- Minimal sharded dataset loaders, decoders, and utils for multi-modal document, image, and text datasets.☆158Updated last year
- Muon: An optimizer for hidden layers in neural networks☆897Updated 2 weeks ago
- Common Python utilities and GitHub Actions in Lightning Ecosystem☆56Updated last week
- Effortless plugin and play Optimizer to cut model training costs by 50%. New optimizer that is 2x faster than Adam on LLMs.☆381Updated last year