awslabs / s3-connector-for-pytorchLinks
The Amazon S3 Connector for PyTorch delivers high throughput for PyTorch training jobs that access and store data in Amazon S3.
☆201Updated last week
Alternatives and similar repositories for s3-connector-for-pytorch
Users that are interested in s3-connector-for-pytorch are comparing it to the libraries listed below
Sorting:
- TorchX is a universal job launcher for PyTorch applications. TorchX is designed to have fast iteration time for training/research and sup…☆411Updated last week
- Scalable and Performant Data Loading☆364Updated this week
- Module, Model, and Tensor Serialization/Deserialization☆286Updated 5 months ago
- ☆278Updated 2 weeks ago
- ☆111Updated last year
- Fault tolerance for PyTorch (HSDP, LocalSGD, DiLoCo, Streaming DiLoCo)☆474Updated 3 weeks ago
- Powering AWS purpose-built machine learning chips. Blazing fast and cost effective, natively integrated into PyTorch and TensorFlow and i…☆573Updated last week
- ☆64Updated last month
- Example code for AWS Neuron SDK developers building inference and training applications☆156Updated 2 weeks ago
- A helper library to connect into Amazon SageMaker with AWS Systems Manager and SSH (Secure Shell)☆258Updated 6 months ago
- Training and inference on AWS Trainium and Inferentia chips.☆258Updated this week
- Container plugin for Slurm Workload Manager☆412Updated 3 weeks ago
- A library to analyze PyTorch traces.☆464Updated last week
- NVIDIA Resiliency Extension is a python package for framework developers and users to implement fault-tolerant features. It improves the …☆255Updated this week
- EFA/NCCL base AMI build Packer and CodeBuild/Pipeline files. Also base Docker build files to enable EFA/NCCL in containers☆43Updated 2 years ago
- Collection of best practices, reference architectures, model training examples and utilities to train large models on AWS.☆386Updated this week
- CUDA checkpoint and restore utility☆406Updated 4 months ago
- ☆57Updated last week
- JetStream is a throughput and memory optimized engine for LLM inference on XLA devices, starting with TPUs (and GPUs in future -- PRs wel…☆403Updated 3 weeks ago
- ☆71Updated 11 months ago
- Deploying EFA in EKS utilizing GPUDirectRDMA where supported☆36Updated last year
- Kubernetes Operator, ansible playbooks, and production scripts for large-scale AIStore deployments on Kubernetes.☆124Updated this week
- ☆321Updated last year
- A CLI tool that helps manage training jobs on the SageMaker HyperPod clusters orchestrated by Amazon EKS☆34Updated last week
- xpk (Accelerated Processing Kit, pronounced x-p-k,) is a software tool to help Cloud developers to orchestrate training jobs on accelerat…☆162Updated this week
- A tool to configure, launch and manage your machine learning experiments.☆215Updated last week
- ☆73Updated last year
- The Triton backend for the PyTorch TorchScript models.☆172Updated 2 weeks ago
- This repository hosts code that supports the testing infrastructure for the PyTorch organization. For example, this repo hosts the logic …☆104Updated this week
- AIPerf is a comprehensive benchmarking tool that measures the performance of generative AI models served by your preferred inference solu…☆112Updated this week