giacbrd / SmartPipelineLinks
A framework for rapid development of robust data pipelines following a simple design pattern
☆27Updated last year
Alternatives and similar repositories for SmartPipeline
Users that are interested in SmartPipeline are comparing it to the libraries listed below
Sorting:
- Create and manage data pipes with Meerschaum.☆144Updated last week
- dagster scikit-learn pipeline example.☆45Updated 2 years ago
- Simple, lightweight, extensible DAG framework for Python with a Kubeflow-like API☆80Updated last year
- A curated list of dagster code snippets for data engineers☆56Updated last year
- DAG based BI-as-code CLI tool. Unlocks a better approach data visualization that integrates seamlessly into the modern data stack.☆49Updated last week
- A monorepo of many Rill example projects☆42Updated last week
- Python ELT Studio, an application for building ELT (and ETL) data flows.☆58Updated 3 years ago
- A guide for leading a data (engineering) team☆64Updated last year
- CLI for running Airbyte sources & destinations locally without Airbyte server☆33Updated last week
- A small Python module containing quick utility functions for standard ETL processes.☆37Updated last week
- Repo demonstrating a Dagster pipeline to generate Neo4j Graph☆22Updated 4 years ago
- Code examples showing flow deployment to various types of infrastructure☆109Updated 2 years ago
- Micro Graph Database for Python Applications☆315Updated last week
- Open Source Data Quality Monitoring.☆158Updated 3 weeks ago
- ☆81Updated 5 months ago
- A portable Datamart and Business Intelligence suite built with Docker, Dagster, dbt, DuckDB and Superset☆237Updated 5 months ago
- dbd is a database prototyping tool that enables data analysts and engineers to quickly load and transform data in SQL databases.☆57Updated 3 years ago
- Swiple enables you to easily observe, understand, validate and improve the quality of your data☆84Updated this week
- Lightning fast OLAP-style point queries on Pandas DataFrames.☆120Updated 8 months ago
- A playground for running duckdb as a stateless query engine over a data lake☆211Updated last year
- 🏗️ Create APIs from CSV files within seconds, using fastapi☆78Updated 4 years ago
- Data processing and modelling framework for automating tasks (incl. Python & SQL transformations).☆120Updated last month
- 🚕 A spreadsheet-like data preparation web app that works over Optimus (Pandas, Dask, cuDF, Dask-cuDF, Spark and Vaex)☆141Updated 2 years ago
- manipulate pandas dataframes from the comfort of your browser☆174Updated 3 years ago
- Deploy a Prefect flow to serverless AWS Lambda function☆35Updated 2 years ago
- DataForge helps data teams write functional transformation pipelines by leveraging software engineering principles☆52Updated this week
- Metamapper is a data discovery and documentation platform for improving how teams understand and interact with their data.☆79Updated this week
- A simple Python-based distributed workflow engine☆57Updated last week
- locopy: Loading/Unloading to Redshift and Snowflake using Python.☆110Updated last week
- A portable Datamart and Business Intelligence suite built with Docker, Mage, dbt, DuckDB and Superset☆53Updated 9 months ago