pditommaso / awesome-pipelineLinks
A curated list of awesome pipeline toolkits inspired by Awesome Sysadmin
☆6,408Updated 4 months ago
Alternatives and similar repositories for awesome-pipeline
Users that are interested in awesome-pipeline are comparing it to the libraries listed below
Sorting:
- A curated list of awesome ETL frameworks, libraries, and software.☆3,428Updated 11 months ago
- Curated list of resources about Apache Airflow☆3,814Updated 10 months ago
- Repository for the CWL standards. Use https://cwl.discourse.group/ for support 😊☆1,470Updated 7 months ago
- Luigi is a Python module that helps you build complex pipelines of batch jobs. It handles dependency resolution, workflow management, vis…☆18,374Updated 2 months ago
- 📚 Parameterize, execute, and analyze notebooks☆6,215Updated last week
- 🦉 Data Versioning and ML Experiments☆14,672Updated this week
- the portable Python dataframe library☆5,923Updated this week
- a curated list of awesome streaming frameworks, applications, etc☆2,861Updated 2 months ago
- A curated list of data engineering tools for software developers☆7,550Updated last week
- Data-Centric Pipelines and Data Versioning☆6,242Updated 5 months ago
- A lightweight opinionated ETL framework, halfway between plain scripts and Apache Airflow☆2,080Updated last year
- Parallel computing with task scheduling☆13,337Updated last week
- A next-generation curated knowledge sharing platform for data scientists and other technical professions.☆5,526Updated 10 months ago
- Agile Data Preparation Workflows made easy with Pandas, Dask, cuDF, Dask-cuDF, Vaex and PySpark☆1,515Updated 7 months ago
- Build, Manage and Deploy AI/ML Systems☆8,966Updated this week
- Quilt is a data mesh for connecting people with actionable data☆1,342Updated last week
- Out-of-Core hybrid Apache Arrow/NumPy DataFrame for Python, ML, visualization and exploration of big tabular data at a billion rows per s…☆8,409Updated 9 months ago
- The fastest ⚡️ way to build data pipelines. Develop iteratively, deploy anywhere. ☁️☆3,594Updated last month
- A curated list of awesome open source workflow engines☆7,209Updated 2 months ago
- ETL best practices with airflow, with examples☆1,339Updated 9 months ago
- Docker Apache Airflow☆3,810Updated 2 years ago
- An orchestration platform for the development, production, and observation of data assets.☆13,567Updated last week
- Kedro is a toolbox for production-ready data science. It uses software engineering best practices to help you create data engineering and…☆10,438Updated this week
- Amundsen is a metadata driven application for improving the productivity of data analysts, data scientists and engineers when interacting…☆4,614Updated 2 weeks ago
- Ready-to-run Docker images containing Jupyter applications☆8,256Updated last week
- Modin: Scale your Pandas workflows by changing a single line of code☆10,223Updated last week
- The Open Source Feature Store for AI/ML☆6,215Updated this week
- A unified interface for distributed computing. Fugue executes SQL, Python, Pandas, and Polars code on Spark, Dask and Ray without any rew…☆2,094Updated 3 months ago
- Build data pipelines, the easy way 🛠️☆4,131Updated 2 years ago
- Extract Transform Load for Python 3.5+☆1,597Updated 2 years ago