uber / petastorm
Petastorm library enables single machine or distributed training and evaluation of deep learning models from datasets in Apache Parquet format. It supports ML frameworks such as Tensorflow, Pytorch, and PySpark and can be used from pure Python code.
☆1,824Updated last year
Alternatives and similar repositories for petastorm:
Users that are interested in petastorm are comparing it to the libraries listed below
- Automated Machine Learning on Kubernetes☆1,550Updated this week
- Open Source ML Model Versioning, Metadata, and Experiment Management☆1,719Updated 7 months ago
- MLeap: Deploy ML Pipelines to Production☆1,515Updated 3 months ago
- Scalable Machine Learning with Dask☆922Updated last month
- Adaptive Experimentation Platform☆2,447Updated this week
- Hopsworks - Data-Intensive AI platform with a Feature Store☆1,216Updated last month
- MLOps Tools For Managing & Orchestrating The Machine Learning LifeCycle☆3,613Updated 2 weeks ago
- Library for exploring and validating machine learning data☆768Updated last week
- Distributed Computing for AI Made Simple☆1,044Updated 2 years ago
- The Open Source Feature Store for AI/ML☆5,874Updated this week
- Extended pickling support for Python objects☆1,716Updated last week
- A low-latency prediction-serving system☆1,413Updated 3 years ago
- PyTorch elastic training☆730Updated 2 years ago
- An MLOps framework to package, deploy, monitor and manage thousands of production machine learning models☆4,488Updated this week
- Model analysis tools for TensorFlow☆1,262Updated last month
- Hummingbird compiles trained ML models into tensor computation for faster inference.☆3,409Updated 2 months ago
- For recording and retrieving metadata associated with ML developer and data scientist workflows.☆640Updated 4 months ago
- cuML - RAPIDS Machine Learning Library☆4,529Updated this week
- TFX is an end-to-end platform for deploying production ML pipelines☆2,136Updated last week
- A uniform interface to run deep learning models from multiple frameworks☆935Updated last year
- High performance model preprocessing library on PyTorch☆650Updated 11 months ago
- TonY is a framework to natively run deep learning frameworks on Apache Hadoop.☆706Updated last year
- A unified interface for distributed computing. Fugue executes SQL, Python, Pandas, and Polars code on Spark, Dask and Ray without any rew…☆2,055Updated 6 months ago
- A model-agnostic visual debugging tool for machine learning☆1,657Updated last month
- NVTabular is a feature engineering and preprocessing library for tabular data designed to quickly and easily manipulate terabyte scale da…☆1,071Updated 6 months ago
- Agile Data Preparation Workflows made easy with Pandas, Dask, cuDF, Dask-cuDF, Vaex and PySpark☆1,497Updated 3 months ago
- Version control for machine learning☆1,660Updated 3 weeks ago
- Input pipeline framework☆987Updated this week
- Bayesian optimization in PyTorch☆3,199Updated this week
- Dataset, streaming, and file system extensions maintained by TensorFlow SIG-IO☆723Updated this week