awslabs / s3-connector-for-pytorchLinks
The Amazon S3 Connector for PyTorch delivers high throughput for PyTorch training jobs that access and store data in Amazon S3.
☆173Updated this week
Alternatives and similar repositories for s3-connector-for-pytorch
Users that are interested in s3-connector-for-pytorch are comparing it to the libraries listed below
Sorting:
- TorchX is a universal job launcher for PyTorch applications. TorchX is designed to have fast iteration time for training/research and sup…☆381Updated this week
- ☆112Updated 6 months ago
- Powering AWS purpose-built machine learning chips. Blazing fast and cost effective, natively integrated into PyTorch and TensorFlow and i…☆533Updated this week
- Module, Model, and Tensor Serialization/Deserialization☆250Updated this week
- EFA/NCCL base AMI build Packer and CodeBuild/Pipeline files. Also base Docker build files to enable EFA/NCCL in containers☆44Updated last year
- Scalable and Performant Data Loading☆291Updated this week
- Easy, fast and very cheap training and inference on AWS Trainium and Inferentia chips.☆235Updated last week
- Fault tolerance for PyTorch (HSDP, LocalSGD, DiLoCo, Streaming DiLoCo)☆377Updated this week
- ☆232Updated this week
- Example code for AWS Neuron SDK developers building inference and training applications☆148Updated this week
- Kubernetes Operator, ansible playbooks, and production scripts for large-scale AIStore deployments on Kubernetes.☆107Updated this week
- Collection of best practices, reference architectures, model training examples and utilities to train large models on AWS.☆331Updated this week
- Container plugin for Slurm Workload Manager☆369Updated this week
- A helper library to connect into Amazon SageMaker with AWS Systems Manager and SSH (Secure Shell)☆249Updated last month
- ☆51Updated last week
- CUDA checkpoint and restore utility☆357Updated 6 months ago
- AWS virtual gpu device plugin provides capability to use smaller virtual gpus for your machine learning inference workloads☆205Updated last year
- Create, List, Update, Delete Amazon EKS clusters. Deploy and manage software on EKS. Run distributed model training and inference example…☆60Updated this week
- ☆65Updated 5 months ago
- ☆72Updated last year
- ☆60Updated last week
- NVIDIA Resiliency Extension is a python package for framework developers and users to implement fault-tolerant features. It improves the …☆196Updated last week
- Infrastructure as code for GPU accelerated managed Kubernetes clusters.☆54Updated 3 months ago
- xpk (Accelerated Processing Kit, pronounced x-p-k,) is a software tool to help Cloud developers to orchestrate training jobs on accelerat…☆136Updated this week
- JetStream is a throughput and memory optimized engine for LLM inference on XLA devices, starting with TPUs (and GPUs in future -- PRs wel…☆365Updated last month
- Create and manage Amazon SageMaker HyperPod clusters, run distributed model training☆24Updated 2 months ago
- ☆142Updated last week
- The Triton backend for the PyTorch TorchScript models.☆158Updated this week
- Deploying EFA in EKS utilizing GPUDirectRDMA where supported☆37Updated 9 months ago
- This Guidance demonstrates how to deploy a machine learning inference architecture on Amazon Elastic Kubernetes Service (Amazon EKS). It …☆44Updated 2 months ago