NVIDIA / multi-storage-clientLinks
Unified high-performance Python client for object and file stores.
☆57Updated 2 weeks ago
Alternatives and similar repositories for multi-storage-client
Users that are interested in multi-storage-client are comparing it to the libraries listed below
Sorting:
- Fault tolerance for PyTorch (HSDP, LocalSGD, DiLoCo, Streaming DiLoCo)☆475Updated this week
- Speed up model training by fixing data loading.☆575Updated this week
- Scalable and Performant Data Loading☆364Updated this week
- A tool to configure, launch and manage your machine learning experiments.☆216Updated this week
- Container plugin for Slurm Workload Manager☆412Updated 3 weeks ago
- NVIDIA Resiliency Extension is a python package for framework developers and users to implement fault-tolerant features. It improves the …☆255Updated this week
- 🚀 Efficiently (pre)training foundation models with native PyTorch features, including FSDP for training and SDPA implementation of Flash…☆279Updated 2 months ago
- Load compute kernels from the Hub☆389Updated last week
- Where GPUs get cooked 👩🍳🔥☆362Updated 2 weeks ago
- TorchX is a universal job launcher for PyTorch applications. TorchX is designed to have fast iteration time for training/research and sup…☆412Updated this week
- A performant, memory-efficient checkpointing library for PyTorch applications, designed with large, complex distributed workloads in mind…☆164Updated 3 weeks ago
- Megatron's multi-modal data loader☆315Updated last week
- A library to analyze PyTorch traces.☆462Updated this week
- Pipeline Parallelism for PyTorch☆784Updated last year
- The Amazon S3 Connector for PyTorch delivers high throughput for PyTorch training jobs that access and store data in Amazon S3.☆203Updated this week
- PyTorch Single Controller☆957Updated this week
- ☆280Updated this week
- torchax is a PyTorch frontend for JAX. It gives JAX the ability to author JAX programs using familiar PyTorch syntax. It also provides JA…☆175Updated this week
- Module, Model, and Tensor Serialization/Deserialization☆286Updated 5 months ago
- LLM KV cache compression made easy☆866Updated last week
- A Python-embedded DSL that makes it easy to write fast, scalable ML kernels with minimal boilerplate.☆739Updated this week
- Pytorch Distributed native training library for LLMs/VLMs with OOTB Hugging Face support☆266Updated this week
- 🚀 Collection of components for development, training, tuning, and inference of foundation models leveraging PyTorch native components.☆219Updated this week
- Easy and lightning fast training of 🤗 Transformers on Habana Gaudi processor (HPU)☆205Updated this week
- TPU inference for vLLM, with unified JAX and PyTorch support.☆228Updated this week
- This repository contains the experimental PyTorch native float8 training UX☆227Updated last year
- KvikIO - High Performance File IO☆240Updated this week
- Provides end-to-end model development pipelines for LLMs and Multimodal models that can be launched on-prem or cloud-native.☆510Updated 9 months ago
- A storage solution for PyTorch tensors with distributed tensor support.☆61Updated this week
- JetStream is a throughput and memory optimized engine for LLM inference on XLA devices, starting with TPUs (and GPUs in future -- PRs wel…☆404Updated last month