Photoroom / datagoLinks
A natively parallel dataloader for Python, written in Rust. Serving data at GB/s speeds, while covering aspect ratio bucketing, crop and resize for image ML workloads.
☆119Updated 3 weeks ago
Alternatives and similar repositories for datago
Users that are interested in datago are comparing it to the libraries listed below
Sorting:
- Framework based on a vector dabase to store, manage and curate large image datasets☆80Updated 2 months ago
- ☆91Updated last year
- Lightweight package that tracks and summarizes code changes using LLMs (Large Language Models)☆34Updated 8 months ago
- Minimal (400 LOC) implementation Maximum (multi-node, FSDP) GPT training☆132Updated last year
- Train vision models using JAX and 🤗 transformers☆100Updated 2 weeks ago
- supporting pytorch FSDP for optimizers☆84Updated 11 months ago
- ☆34Updated last year
- An implementation of PSGD Kron second-order optimizer for PyTorch☆97Updated 3 months ago
- ☆310Updated last year
- Focused on fast experimentation and simplicity☆75Updated 10 months ago
- An implementation of the Llama architecture, to instruct and delight☆21Updated 5 months ago
- ☆40Updated last year
- Scalable and Performant Data Loading☆335Updated this week
- ☆50Updated last year
- WIP☆93Updated last year
- ☆86Updated 4 months ago
- Fast, Modern, and Low Precision PyTorch Optimizers☆116Updated 2 months ago
- Transformer with Mu-Parameterization, implemented in Jax/Flax. Supports FSDP on TPU pods.☆32Updated 5 months ago
- Minimal sharded dataset loaders, decoders, and utils for multi-modal document, image, and text datasets.☆159Updated last year
- A place to store reusable transformer components of my own creation or found on the interwebs☆62Updated last month
- PyTorch centric eager mode debugger☆48Updated 11 months ago
- Efficient optimizers☆275Updated last week
- DeMo: Decoupled Momentum Optimization☆197Updated 11 months ago
- Automatically take good care of your preemptible TPUs☆37Updated 2 years ago
- ☆53Updated last year
- Code and weights for the paper "Cluster and Predict Latents Patches for Improved Masked Image Modeling"☆123Updated 7 months ago
- ☆21Updated 8 months ago
- Solve puzzles. Learn CUDA.☆64Updated last year
- j1-micro (1.7B) & j1-nano (600M) are absurdly tiny but mighty reward models.☆97Updated 4 months ago
- research impl of Native Sparse Attention (2502.11089)☆63Updated 9 months ago