pracdata / duckdb-pipeline
Demonstrating the capabilities of DuckDB as a transformation engine for data lakes
☆24Updated 7 months ago
Alternatives and similar repositories for duckdb-pipeline:
Users that are interested in duckdb-pipeline are comparing it to the libraries listed below
- Using DuckDB with AWS Lambda to process Delta Lake data☆26Updated 3 months ago
- Yet Another (Spark) ETL Framework☆21Updated last year
- A portable Datamart and Business Intelligence suite built with Docker, sqlmesh + dbtcore, DuckDB and Superset☆50Updated 6 months ago
- Open Data Stack Platform: a collection of projects and pipelines built with open data stack tools for scalable, observable data platform…☆16Updated 2 weeks ago
- Unity Catalog UI☆40Updated 8 months ago
- A portable Datamart and Business Intelligence suite built with Docker, Mage, dbt, DuckDB and Superset☆53Updated 6 months ago
- A write-audit-publish implementation on a data lake without the JVM☆46Updated 8 months ago
- A portable Datamart and Business Intelligence suite built with Docker, Airflow, dbt, PostgreSQL and Superset☆41Updated 5 months ago
- Cost Efficient Data Pipelines with DuckDB☆52Updated 9 months ago
- Nicely modeled data built on the Github Archive.☆65Updated last year
- ☆17Updated 9 months ago
- Contribute to dlt verified sources 🔥☆84Updated this week
- Repo for orienting dbt users to the Dagster asset framework☆54Updated 2 years ago
- Example files used in the DuckDB - Unity Catalog blog☆10Updated 5 months ago
- A simple and easy to use Data Quality (DQ) tool built with Python.☆50Updated last year
- Dagster SQLMesh Adapter☆52Updated last week
- Example Dagster Cloud code for the Hooli Data Engineering organization.☆1Updated this week
- Arcane Insight is a data analytics project designed to harness the power of SQLMesh & DuckDB to collect, transform, and analyze data from…☆33Updated 3 months ago
- Data Quality and Observability platform for the whole data lifecycle, from profiling new data sources to full automation with Data Observ…☆145Updated last month
- Data-aware orchestration with dagster, dbt, and airbyte☆31Updated 2 years ago
- New generation opensource data stack☆67Updated 2 years ago
- ☆34Updated 3 weeks ago
- Open Data Stack Projects: Examples of End to End Data Engineering Projects☆83Updated last year
- Department of Education (DOE) for New South Wales (AUS) data stack in a box☆32Updated 5 months ago
- end-to-end data engineering project to get insights from PyPi using python, duckdb, MotherDuck & Evidence☆197Updated this week
- Declarative text based tool for data analysts and engineers to extract, load, transform and orchestrate their data pipelines.☆108Updated this week
- Full stack data engineering tools and infrastructure set-up☆52Updated 4 years ago
- Sample code to collect Apache Iceberg metrics for table monitoring☆26Updated 8 months ago
- A platform and cloud-based service for data sharing based on the Delta Sharing protocol.☆21Updated 10 months ago
- A DataOps framework for building a lakehouse.☆50Updated last week