pditommaso / awesome-pipelineLinks
A curated list of awesome pipeline toolkits inspired by Awesome Sysadmin
☆6,391Updated 3 months ago
Alternatives and similar repositories for awesome-pipeline
Users that are interested in awesome-pipeline are comparing it to the libraries listed below
Sorting:
- A curated list of awesome ETL frameworks, libraries, and software.☆3,416Updated 11 months ago
- Curated list of resources about Apache Airflow☆3,799Updated 10 months ago
- 📚 Parameterize, execute, and analyze notebooks☆6,201Updated 2 months ago
- Repository for the CWL standards. Use https://cwl.discourse.group/ for support 😊☆1,469Updated 6 months ago
- Luigi is a Python module that helps you build complex pipelines of batch jobs. It handles dependency resolution, workflow management, vis…☆18,349Updated last month
- Data-Centric Pipelines and Data Versioning☆6,236Updated 4 months ago
- A lightweight opinionated ETL framework, halfway between plain scripts and Apache Airflow☆2,079Updated last year
- A next-generation curated knowledge sharing platform for data scientists and other technical professions.☆5,523Updated 9 months ago
- Always know what to expect from your data.☆10,488Updated this week
- NumPy and Pandas interface to Big Data☆3,197Updated last year
- dbt enables data analysts and engineers to transform their data using the same practices that software engineers use to build application…☆11,011Updated this week
- A Grammar of Graphics for Python☆4,255Updated this week
- A curated list of data engineering tools for software developers☆7,488Updated this week
- Parallel computing with task scheduling☆13,291Updated this week
- Quilt is a data mesh for connecting people with actionable data☆1,341Updated this week
- Declarative visualization library for Python☆9,851Updated 3 weeks ago
- Out-of-Core hybrid Apache Arrow/NumPy DataFrame for Python, ML, visualization and exploration of big tabular data at a billion rows per s…☆8,399Updated 8 months ago
- Feather: fast, interoperable binary data frame storage for Python, R, and more powered by Apache Arrow☆2,746Updated 3 years ago
- the portable Python dataframe library☆5,866Updated this week
- Construct Apache Airflow DAGs Declaratively via YAML configuration files☆1,310Updated this week
- ETL best practices with airflow, with examples☆1,337Updated 9 months ago
- Build data pipelines, the easy way 🛠️☆4,124Updated 2 years ago
- Python Extract Transform and Load Tables of Data☆1,273Updated last month
- A scalable, efficient, cross-platform (Linux/macOS) and easy-to-use workflow engine in pure Python.☆913Updated this week
- A curated list of awesome Jupyter projects, libraries and resources☆4,279Updated this week
- Extract Transform Load for Python 3.5+☆1,592Updated 2 years ago
- Amundsen is a metadata driven application for improving the productivity of data analysts, data scientists and engineers when interacting…☆4,604Updated this week
- A DSL for data-driven computational pipelines☆2,965Updated this week
- Agile Data Preparation Workflows made easy with Pandas, Dask, cuDF, Dask-cuDF, Vaex and PySpark☆1,513Updated 6 months ago
- A functional standard library for Python.☆4,909Updated this week