GoogleCloudPlatform / dataflux-client-pythonLinks
This is the git repository for the Dataflux Python client library, providing fast listing and download of small files from GCS in Python. Also see https://github.com/GoogleCloudPlatform/dataflux-pytorch.
☆19Updated 2 weeks ago
Alternatives and similar repositories for dataflux-client-python
Users that are interested in dataflux-client-python are comparing it to the libraries listed below
Sorting:
- High-performance Python librarys for connecting AI/ML frameworks with OSS storage.☆23Updated 3 weeks ago
- MLPerf™ Storage Benchmark Suite☆147Updated this week
- ☆29Updated 4 years ago
- Ephemeral distributed filesystem build up from the local storage of several nodes. It is an evolution of AdaFS done inside the NGIO proje…☆36Updated 3 years ago
- Health checks for Azure N- and H-series VMs.☆42Updated last month
- GeminiFS: A Companion File System for GPUs☆32Updated 3 months ago
- ☆24Updated last year
- Fine-grained GPU sharing primitives☆141Updated 5 years ago
- NCCL Profiling Kit☆135Updated 11 months ago
- This repository contains experimental tools we developed to forecast a clusters' resource (CPU or memory) usage.☆40Updated 4 years ago
- Magnum IO community repo☆95Updated 3 weeks ago
- ☆24Updated last year
- NVIDIA Inference Xfer Library (NIXL)☆365Updated this week
- Speed up fsspec data access with Alluxio distributed caching.☆14Updated 2 months ago
- ☆53Updated 4 years ago
- Code for "Heterogenity-Aware Cluster Scheduling Policies for Deep Learning Workloads", which appeared at OSDI 2020☆128Updated 10 months ago
- DPU-Powered File System Virtualization over virtio-fs☆70Updated last year
- FireFlyer Record file format, writer and reader for DL training samples.☆224Updated 2 years ago
- verbs profiling library☆22Updated last year
- RDMA and SHARP plugins for nccl library☆195Updated last month
- Artifacts for our NSDI'23 paper TGS☆75Updated 11 months ago
- Medusa: Accelerating Serverless LLM Inference with Materialization [ASPLOS'25]☆22Updated 3 weeks ago
- NCCL Fast Socket is a transport layer plugin to improve NCCL collective communication performance on Google Cloud.☆116Updated last year
- An I/O benchmark for deep Learning applications☆87Updated last week
- [HotStorage '24] Can ZNS SSDs be Better Storage Devices for Persistent Cache?☆12Updated 11 months ago
- Hooked CUDA-related dynamic libraries by using automated code generation tools.☆156Updated last year
- NVIDIA GPUDirect Storage Driver☆248Updated last month
- An RDMA-enabled Distributed Persistent Memory File System☆157Updated 7 years ago
- Thunder Research Group's Collective Communication Library☆37Updated last year
- Rcmp: Reconstructing RDMA-based Memory Disaggregation via CXL☆55Updated last year