Lightning-AI / litDataLinks
Transform datasets at scale. Optimize datasets for fast AI model training.
☆485Updated this week
Alternatives and similar repositories for litData
Users that are interested in litData are comparing it to the libraries listed below
Sorting:
- Scalable and Performant Data Loading☆269Updated this week
- Thunder gives you PyTorch models superpowers for training and inference. Unlock out-of-the-box optimizations for performance, memory and …☆1,357Updated this week
- TensorDict is a pytorch dedicated tensor container.☆925Updated last week
- PyTorch per step fault tolerance (actively under development)☆302Updated this week
- Helpful tools and examples for working with flex-attention☆811Updated this week
- A subset of PyTorch's neural network modules, written in Python using OpenAI's Triton.☆546Updated this week
- For optimization algorithm research and development.☆518Updated this week
- A library that contains a rich collection of performant PyTorch model metrics, a simple interface to create new metrics, a toolkit to fac…☆233Updated 4 months ago
- Universal Tensor Operations in Einstein-Inspired Notation for Python.☆374Updated last month
- Best practices & guides on how to write distributed pytorch training code☆433Updated 3 months ago
- Implementation of 💍 Ring Attention, from Liu et al. at Berkeley AI, in Pytorch☆514Updated 3 weeks ago
- Annotated version of the Mamba paper☆482Updated last year
- A PyTorch repo for data loading and utilities to be shared by the PyTorch domain libraries.☆1,201Updated this week
- Library for reading and processing ML training data.☆447Updated this week
- ☆348Updated 3 weeks ago
- Legible, Scalable, Reproducible Foundation Models with Named Tensors and Jax☆586Updated last week
- Implementation of Diffusion Transformer (DiT) in JAX☆276Updated 11 months ago
- PyTorch video decoding☆572Updated this week
- Create powerful Hydra applications without the yaml files and boilerplate code.☆385Updated this week
- ☆303Updated 11 months ago
- Common Python utilities and GitHub Actions in Lightning Ecosystem☆56Updated this week
- Named tensors with first-class dimensions for PyTorch☆331Updated last year
- Tensors, for human consumption☆1,252Updated last week
- TorchFix - a linter for PyTorch-using code with autofix support☆141Updated 3 months ago
- The AdEMAMix Optimizer: Better, Faster, Older.☆183Updated 8 months ago
- Minimal sharded dataset loaders, decoders, and utils for multi-modal document, image, and text datasets.☆157Updated last year
- [ICLR 2025] Samba: Simple Hybrid State Space Models for Efficient Unlimited Context Language Modeling☆876Updated last month
- PyTorch native quantization and sparsity for training and inference☆2,072Updated this week
- Muon: An optimizer for hidden layers in neural networks☆678Updated last week
- The merlin dataloader lets you rapidly load tabular data for training deep leaning models with TensorFlow, PyTorch or JAX☆418Updated last year