pditommaso / awesome-pipeline
A curated list of awesome pipeline toolkits inspired by Awesome Sysadmin
☆6,257Updated last month
Alternatives and similar repositories for awesome-pipeline:
Users that are interested in awesome-pipeline are comparing it to the libraries listed below
- A curated list of awesome ETL frameworks, libraries, and software.☆3,326Updated 6 months ago
- the portable Python dataframe library☆5,466Updated this week
- Luigi is a Python module that helps you build complex pipelines of batch jobs. It handles dependency resolution, workflow management, vis…☆18,045Updated last week
- A lightweight opinionated ETL framework, halfway between plain scripts and Apache Airflow☆2,081Updated last year
- Curated list of resources about Apache Airflow☆3,726Updated 5 months ago
- 📚 Parameterize, execute, and analyze notebooks☆6,066Updated 3 weeks ago
- Data-Centric Pipelines and Data Versioning☆6,199Updated this week
- Declarative visualization library for Python☆9,539Updated last week
- Always know what to expect from your data.☆10,150Updated this week
- a curated list of awesome streaming frameworks, applications, etc☆2,747Updated 3 weeks ago
- A curated list of data engineering tools for software developers☆6,995Updated 3 months ago
- Out-of-Core hybrid Apache Arrow/NumPy DataFrame for Python, ML, visualization and exploration of big tabular data at a billion rows per s…☆8,330Updated 3 months ago
- An orchestration platform for the development, production, and observation of data assets.☆12,388Updated this week
- Jupyter Notebooks as Markdown Documents, Julia, Python or R scripts☆6,723Updated last month
- Repository for the CWL standards. Use https://cwl.discourse.group/ for support 😊☆1,457Updated last month
- A curated list of awesome open source workflow engines☆6,674Updated 3 weeks ago
- Apache Airflow - A platform to programmatically author, schedule, and monitor workflows☆38,453Updated this week
- A logical, reasonably standardized, but flexible project structure for doing and sharing data science work.☆8,517Updated 2 weeks ago
- A next-generation curated knowledge sharing platform for data scientists and other technical professions.☆5,497Updated 4 months ago
- Computing with Python functions.☆3,953Updated this week
- Quickly and accurately render even the largest data.☆3,367Updated last week
- Prefect is a workflow orchestration framework for building resilient data pipelines in Python.☆18,131Updated this week
- Kedro is a toolbox for production-ready data science. It uses software engineering best practices to help you create data engineering and…☆10,131Updated this week
- A curated list of awesome data visualization libraries and resources.☆3,867Updated last year
- Quilt is a data mesh for connecting people with actionable data☆1,330Updated this week
- Run your code in the cloud, with technology so advanced, it feels like magic!☆2,597Updated this week
- Feather: fast, interoperable binary data frame storage for Python, R, and more powered by Apache Arrow☆2,738Updated 3 years ago
- dbt enables data analysts and engineers to transform their data using the same practices that software engineers use to build application…☆10,275Updated this week
- A cross-platform command-line utility that creates projects from cookiecutters (project templates), e.g. Python package projects, C proje…☆22,962Updated last week
- Parallel computing with task scheduling☆12,883Updated this week