A Ray-based data loader with per-epoch shuffling and configurable pipelining, for shuffling and loading training data for distributed training of machine learning models.
☆18Jan 5, 2023Updated 3 years ago
Alternatives and similar repositories for ray_shuffling_data_loader
Users that are interested in ray_shuffling_data_loader are comparing it to the libraries listed below
Sorting:
- Distributed ML Optimizer☆35Jul 28, 2021Updated 4 years ago
- Deadline-based hyperparameter tuning on RayTune.☆32Jan 16, 2020Updated 6 years ago
- BytePS examples (Vision, NLP, GAN, etc)☆19Nov 24, 2022Updated 3 years ago
- Staging area for ongoing enhancements to Ray focused on improving integration with AWS and other Amazon technologies.☆67Dec 29, 2023Updated 2 years ago
- ☆21Sep 24, 2018Updated 7 years ago
- Codebase for the paper "Beyond BatchNorm: Towards a Unified Understanding of Normalization in Deep Learning"☆17Jul 12, 2021Updated 4 years ago
- ☆16Dec 4, 2019Updated 6 years ago
- High performance NCCL plugin for Bagua.☆15Sep 15, 2021Updated 4 years ago
- Port of Detectron2 to train/deploy model on Amazon Sagemaker☆16Mar 5, 2021Updated 5 years ago
- Terraform Crusoe Cloud provider☆26Mar 4, 2026Updated 2 weeks ago
- Elastic Deep Learning for deep learning framework on Kubernetes☆175Jul 5, 2023Updated 2 years ago
- Resource-adaptive cluster scheduler for deep learning training.☆453Mar 5, 2023Updated 3 years ago
- Terraform module to create an Elastic Kubernetes (EKS) cluster and associated worker instances on AWS☆14Aug 26, 2020Updated 5 years ago
- "Learning Rate Dropout" in PyTorch☆34Dec 6, 2019Updated 6 years ago
- VB Diarization with Eigenvoice and HMM Priors, refactored☆15Jul 27, 2021Updated 4 years ago
- ☆16Nov 2, 2025Updated 4 months ago
- Core communication lib for Bagua.☆48Sep 15, 2021Updated 4 years ago
- ☆11Oct 21, 2024Updated last year
- 非沪籍高校毕业生留沪各项流程汇总☆17Jan 24, 2018Updated 8 years ago
- ☆16Jul 29, 2025Updated 7 months ago
- Mimir is a new implementation of MapReduce over MPI. Mimir inherits the core principles of existing MapReduce frameworks, such as MR-MPI,…☆21Nov 12, 2018Updated 7 years ago
- Elastic computing platform☆30Updated this week
- LightGBM on Ray☆51Feb 4, 2024Updated 2 years ago
- A boilerplate for seamlessly integrating PyTorch's Distributed Data Parallel (DDP) with SLURM job scheduling and Weights and Biases. Kick…☆10Aug 21, 2023Updated 2 years ago
- ☆14Oct 18, 2023Updated 2 years ago
- Ray based map function for Deap with examples. Automatically batches iterator to multiple workers for many processors or clusters.☆16Jul 26, 2020Updated 5 years ago
- RayDP provides simple APIs for running Spark on Ray and integrating Spark with AI libraries.☆370Updated this week
- My Python solutions to Udacity's CS262 Programming Languages assignments☆16Jun 28, 2012Updated 13 years ago
- Kubernetes Operator for AI and Bigdata Elastic Training☆91Jan 10, 2025Updated last year
- Distributed optimization framework with parameter server☆23Jun 14, 2015Updated 10 years ago
- The tool is to fetch github discussion as blog post and deploy to vercel or github page☆12Jan 16, 2023Updated 3 years ago
- Specification of a protocol for payment channels in TLA+☆15May 20, 2025Updated 10 months ago
- Some crazy experiments☆35Sep 3, 2025Updated 6 months ago
- Memory footprint reduction for transformer models☆11Jan 24, 2023Updated 3 years ago
- A simple tool for parsing the profile.json file of mxnet☆14Aug 1, 2018Updated 7 years ago
- repository for notes and data from machine learning studies☆13Dec 16, 2019Updated 6 years ago
- Pytorch Lightning Distributed Accelerators using Ray☆216Nov 3, 2023Updated 2 years ago
- ☆12May 3, 2020Updated 5 years ago
- Deep Metric Transfer for Label Propagation with Limited Annotated Data☆50Jun 3, 2023Updated 2 years ago