pracdata / duckdb-pipeline
Demonstrating the capabilities of DuckDB as a transformation engine for data lakes
☆21Updated 4 months ago
Alternatives and similar repositories for duckdb-pipeline:
Users that are interested in duckdb-pipeline are comparing it to the libraries listed below
- A portable Datamart and Business Intelligence suite built with Docker, sqlmesh + dbtcore, DuckDB and Superset☆47Updated 3 months ago
- Sample code to collect Apache Iceberg metrics for table monitoring☆24Updated 6 months ago
- ☆17Updated 6 months ago
- Unity Catalog UI☆39Updated 5 months ago
- Delta Lake Documentation☆48Updated 8 months ago
- Example Dagster Cloud code for the Hooli Data Engineering organization.☆81Updated last month
- A write-audit-publish implementation on a data lake without the JVM☆46Updated 6 months ago
- Yet Another (Spark) ETL Framework☆18Updated last year
- Nicely modeled data built on the Github Archive.☆63Updated 11 months ago
- Code snippets for Data Engineering Design Patterns book☆69Updated 2 weeks ago
- Contribute to dlt verified sources 🔥☆80Updated 3 weeks ago
- Full stack data engineering tools and infrastructure set-up☆49Updated 4 years ago
- end-to-end data engineering project to get insights from PyPi using python, duckdb, MotherDuck & Evidence☆183Updated this week
- A Python package that creates fine-grained dbt tasks on Apache Airflow☆64Updated 4 months ago
- Using DuckDB with AWS Lambda to process Delta Lake data☆20Updated 3 weeks ago
- Pythonic Iceberg REST Catalog☆72Updated 5 months ago
- Pytest plugin for dbt core☆58Updated last month
- The bridge to effortless multi-engine data applications, currently supports Snowflake ❄️ and DuckDB 🦆☆157Updated this week
- ☆74Updated 4 months ago
- Cost Efficient Data Pipelines with DuckDB☆49Updated 6 months ago
- ☆15Updated last year
- A Table format agnostic data sharing framework☆38Updated last year
- Alto is a versatile data integration tool that allows you to easily run Singer plugins, build and cache PEX files encapsulating those plu…☆59Updated last year
- dbt (data build tool) projects targeting AWS analytics services (redshift, glue, emr, athena) and open table formats☆29Updated last year
- Repo for orienting dbt users to the Dagster asset framework☆53Updated 2 years ago
- Pythonic Programming Framework to orchestrate jobs in Databricks Workflow☆199Updated last week
- Open Data Stack Projects: Examples of End to End Data Engineering Projects☆75Updated last year
- A platform and cloud-based service for data sharing based on the Delta Sharing protocol.☆21Updated 8 months ago
- build dw with dbt☆36Updated 3 months ago