webdataset / tarp
Fast and simple stream processing of files in tar files, useful for deep learning, big data, and many other applications.
☆127Updated last year
Alternatives and similar repositories for tarp:
Users that are interested in tarp are comparing it to the libraries listed below
- A small demonstration of using WebDataset with ImageNet and PyTorch Lightning☆74Updated last year
- gpu tester detects broken and slow gpus in a cluster☆67Updated 2 years ago
- A python library that provides common I/O interface across different storage backends.☆140Updated last month
- An unopinionated replacement for PyTorch's Dataset and ImageFolder, that handles Tar archives☆76Updated 2 years ago
- Examples for the WebDataset PyTorch Dataset Library☆47Updated 3 years ago
- Simple and efficient RevNet-Library for PyTorch with XLA and DeepSpeed support and parameter offload☆126Updated 2 years ago
- CLOOB training (JAX) and inference (JAX and PyTorch)☆70Updated 2 years ago
- Implementation of Hourglass Transformer, in Pytorch, from Google and OpenAI☆84Updated 3 years ago
- Named tensors with first-class dimensions for PyTorch☆321Updated last year
- ☆109Updated 3 years ago
- Utilities for sequential processing of tar files.☆24Updated 3 years ago
- Optimized library for large-scale extraction of frames and audio from video.☆203Updated last year
- An open source implementation of CLIP.☆32Updated 2 years ago
- Contrastive Language-Image Pretraining☆142Updated 2 years ago
- Simple python template☆40Updated 10 months ago
- JAX implementation ViT-VQGAN☆82Updated 2 years ago
- ☆27Updated 3 years ago
- EfficientNet, MobileNetV3, MobileNetV2, MixNet, etc in JAX w/ Flax Linen and Objax☆127Updated last year
- Texture mapping with variational auto-encoders☆40Updated 3 years ago
- ☆25Updated 3 years ago
- A criticism of a recent paper on buggy image downsampling methods in popular image processing and deep learning libraries.☆72Updated 2 years ago
- DiffQ performs differentiable quantization using pseudo quantization noise. It can automatically tune the number of bits used per weight …☆235Updated last year
- Implementation of fused cosine similarity attention in the same style as Flash Attention☆210Updated 2 years ago
- ☆156Updated 2 years ago
- [Prototype] Tools for the concurrent manipulation of variably sized Tensors.☆253Updated 2 years ago
- ☆12Updated 3 years ago
- Re-implementation of 'Grokking: Generalization beyond overfitting on small algorithmic datasets'☆38Updated 3 years ago
- ☆68Updated last year
- Implementation of the specific Transformer architecture from PaLM - Scaling Language Modeling with Pathways - in Jax (Equinox framework)☆186Updated 2 years ago
- A selection of neural network models ported from torchvision for JAX & Flax.☆44Updated 4 years ago