pditommaso / awesome-pipelineLinks
A curated list of awesome pipeline toolkits inspired by Awesome Sysadmin
☆6,477Updated 3 weeks ago
Alternatives and similar repositories for awesome-pipeline
Users that are interested in awesome-pipeline are comparing it to the libraries listed below
Sorting:
- A curated list of awesome ETL frameworks, libraries, and software.☆3,486Updated last year
- Curated list of resources about Apache Airflow☆3,856Updated last year
- Repository for the CWL standards. Use https://cwl.discourse.group/ for support 😊☆1,473Updated 11 months ago
- 📚 Parameterize, execute, and analyze notebooks☆6,315Updated this week
- A lightweight opinionated ETL framework, halfway between plain scripts and Apache Airflow☆2,086Updated last year
- Luigi is a Python module that helps you build complex pipelines of batch jobs. It handles dependency resolution, workflow management, vis…☆18,557Updated 6 months ago
- A next-generation curated knowledge sharing platform for data scientists and other technical professions.☆5,536Updated last year
- the portable Python dataframe library☆6,215Updated last week
- Always know what to expect from your data.☆10,929Updated this week
- Data-Centric Pipelines and Data Versioning☆6,267Updated 9 months ago
- Parallel computing with task scheduling☆13,603Updated this week
- Actively curated list of awesome BI tools. PRs welcome!☆2,240Updated last year
- Out-of-Core hybrid Apache Arrow/NumPy DataFrame for Python, ML, visualization and exploration of big tabular data at a billion rows per s…☆8,450Updated 3 weeks ago
- A unified interface for distributed computing. Fugue executes SQL, Python, Pandas, and Polars code on Spark, Dask and Ray without any rew…☆2,123Updated 2 weeks ago
- An orchestration platform for the development, production, and observation of data assets.☆14,455Updated this week
- Kedro is a toolbox for production-ready data science. It uses software engineering best practices to help you create data engineering and…☆10,640Updated this week
- A curated list of awesome Jupyter projects, libraries and resources☆4,465Updated this week
- Agile Data Preparation Workflows made easy with Pandas, Dask, cuDF, Dask-cuDF, Vaex and PySpark☆1,527Updated 11 months ago
- Quickly and accurately render even the largest data.☆3,482Updated last month
- Build, Manage and Deploy AI/ML Systems☆9,627Updated last week
- Build data pipelines, the easy way 🛠️☆4,145Updated 2 years ago
- This is the development home of the workflow management system Snakemake. For general information, see☆2,637Updated last week
- Amundsen is a metadata driven application for improving the productivity of data analysts, data scientists and engineers when interacting…☆4,688Updated 3 weeks ago
- ETL best practices with airflow, with examples☆1,351Updated last year
- Extract Transform Load for Python 3.5+☆1,605Updated 2 years ago
- dbt enables data analysts and engineers to transform their data using the same practices that software engineers use to build application…☆11,861Updated this week
- A curated list of awesome open source workflow engines☆7,471Updated 2 months ago
- A curated list of awesome big data frameworks, ressources and other awesomeness.☆14,032Updated 9 months ago
- Quilt is a data mesh for connecting people with actionable data☆1,352Updated last week
- A curated list of awesome Dash (plotly) resources☆2,237Updated 10 months ago