Lightning-AI / litDataLinks
Transform datasets at scale. Optimize datasets for fast AI model training.
☆505Updated last week
Alternatives and similar repositories for litData
Users that are interested in litData are comparing it to the libraries listed below
Sorting:
- Scalable and Performant Data Loading☆288Updated last week
- PyTorch compiler that accelerates training and inference. Get built-in optimizations for performance, memory, parallelism, and easily wri…☆1,375Updated this week
- For optimization algorithm research and development.☆521Updated this week
- TensorDict is a pytorch dedicated tensor container.☆937Updated last week
- A library that contains a rich collection of performant PyTorch model metrics, a simple interface to create new metrics, a toolkit to fac…☆235Updated 6 months ago
- Fault tolerance for PyTorch (HSDP, LocalSGD, DiLoCo, Streaming DiLoCo)☆359Updated 2 weeks ago
- Universal Tensor Operations in Einstein-Inspired Notation for Python.☆385Updated 3 months ago
- Helpful tools and examples for working with flex-attention☆876Updated this week
- PyTorch Single Controller☆318Updated this week
- Common Python utilities and GitHub Actions in Lightning Ecosystem☆56Updated last week
- A subset of PyTorch's neural network modules, written in Python using OpenAI's Triton.☆561Updated last month
- PyTorch media decoding and encoding☆624Updated this week
- Annotated version of the Mamba paper☆486Updated last year
- ☆303Updated last year
- Official Implementation of "ADOPT: Modified Adam Can Converge with Any β2 with the Optimal Rate"☆426Updated 7 months ago
- Create powerful Hydra applications without the yaml files and boilerplate code.☆388Updated 2 weeks ago
- Best practices & guides on how to write distributed pytorch training code☆450Updated 4 months ago
- PyTorch native quantization and sparsity for training and inference☆2,168Updated this week
- A tool to configure, launch and manage your machine learning experiments.☆171Updated this week
- [ICLR 2025] Samba: Simple Hybrid State Space Models for Efficient Unlimited Context Language Modeling☆888Updated 2 months ago
- The AdEMAMix Optimizer: Better, Faster, Older.☆183Updated 10 months ago
- A pytorch quantization backend for optimum☆962Updated last week
- Minimal sharded dataset loaders, decoders, and utils for multi-modal document, image, and text datasets.☆158Updated last year
- TorchX is a universal job launcher for PyTorch applications. TorchX is designed to have fast iteration time for training/research and sup…☆370Updated 2 weeks ago
- Implementation of 💍 Ring Attention, from Liu et al. at Berkeley AI, in Pytorch☆529Updated 2 months ago
- A PyTorch repo for data loading and utilities to be shared by the PyTorch domain libraries.☆1,211Updated this week
- Efficient optimizers☆232Updated last week
- A Jax-based library for building transformers, includes implementations of GPT, Gemma, LlaMa, Mixtral, Whisper, SWin, ViT and more.☆290Updated 10 months ago
- TensorHue is a Python library that allows you to visualize tensors right in your console, making understanding and debugging tensor conte…☆118Updated 4 months ago
- Where GPUs get cooked 👩🍳🔥☆236Updated 4 months ago