GoogleCloudPlatform / dataflux-client-pythonLinks
This is the git repository for the Dataflux Python client library, providing fast listing and download of small files from GCS in Python. Also see https://github.com/GoogleCloudPlatform/dataflux-pytorch.
☆22Updated 8 months ago
Alternatives and similar repositories for dataflux-client-python
Users that are interested in dataflux-client-python are comparing it to the libraries listed below
Sorting:
- MLPerf® Storage Benchmark Suite☆173Updated last week
- High-performance Python librarys for connecting AI/ML frameworks with OSS storage.☆25Updated 4 months ago
- Speed up fsspec data access with Alluxio distributed caching.☆18Updated last month
- Tracking Ray Enhancement Proposals☆63Updated last month
- ☆16Updated 2 weeks ago
- ☆109Updated 3 years ago
- An RDMA-enabled Distributed Persistent Memory File System☆160Updated 8 years ago
- Addendum to FAST15 Paper: Analysis of the ECMWF Storage Landscape☆13Updated 10 years ago
- FireFlyer Record file format, writer and reader for DL training samples.☆238Updated 3 years ago
- DAOS Storage Stack (client libraries, storage engine, control plane)☆911Updated this week
- USENIX FAST 2021, "Exploiting Combined Locality for Wide-Stripe Erasure Coding in Distributed Storage"☆34Updated 4 years ago
- High Fidelity Workload Replay Engine☆17Updated 8 years ago
- NVIDIA Inference Xfer Library (NIXL)☆864Updated this week
- NVIDIA GPUDirect Storage Driver☆331Updated last month
- RayDP provides simple APIs for running Spark on Ray and integrating Spark with AI libraries.☆360Updated this week
- A tutorial on RDMA based programming using code examples☆597Updated 6 years ago
- ☆110Updated last week
- ☆34Updated 6 months ago
- Sample code from thegeekinthecorner.com☆280Updated 5 years ago
- A framework to understand RDMA☆406Updated 2 years ago
- An I/O benchmark for deep Learning applications☆102Updated last month
- Magnum IO community repo☆109Updated 2 months ago
- Open-Channel SSD emulator using memory☆21Updated 8 years ago
- IO500 Storage Benchmark source code☆128Updated 3 months ago
- ☆15Updated 2 years ago
- λFS: an elastic, high-performance, serverless-function-based metadata service for large-scale distributed file systems (ACM ASPLOS'23)☆14Updated 10 months ago
- Snowflake dataset containing statistics for 70 million queries over 14 day period☆115Updated 4 years ago
- High Performance KV Cache Store for LLM☆45Updated last week
- Apache Iceberg C++☆188Updated last week
- A validation and profiling tool for AI infrastructure☆359Updated last week