moritzkoerber / covid-19-data-engineering-pipeline
A Covid-19 data pipeline on AWS featuring PySpark/Glue, Docker, Great Expectations, Airflow, and Redshift, templated in CloudFormation and CDK, deployable via Github Actions.
☆23Updated last year
Alternatives and similar repositories for covid-19-data-engineering-pipeline:
Users that are interested in covid-19-data-engineering-pipeline are comparing it to the libraries listed below
- Code for my "Efficient Data Processing in SQL" book.☆56Updated 5 months ago
- build dw with dbt☆35Updated 3 months ago
- Open Data Stack Projects: Examples of End to End Data Engineering Projects☆73Updated last year
- ☆31Updated last month
- End to end data engineering project☆53Updated 2 years ago
- A simple and easy to use Data Quality (DQ) tool built with Python.☆49Updated last year
- ☆18Updated 5 months ago
- ☆73Updated 3 months ago
- Step-by-step tutorial on building a Kimball dimensional model with dbt☆122Updated 6 months ago
- A template for dockerized dbt-Core projects with VS Code Dev Containers.☆20Updated 2 years ago
- A Python package that creates fine-grained dbt tasks on Apache Airflow☆63Updated 4 months ago
- Code to demonstrate data engineering metadata & logging best practices☆15Updated 10 months ago
- A portable Datamart and Business Intelligence suite built with Docker, sqlmesh + dbtcore, DuckDB and Superset☆48Updated 2 months ago
- Project for "Data pipeline design patterns" blog.☆43Updated 5 months ago
- Template for Data Engineering and Data Pipeline projects☆106Updated 2 years ago
- dbt Project for Rapid Onboarding instructors to use in instruction and learners to reference throughout the course.☆23Updated this week
- Cost Efficient Data Pipelines with DuckDB☆48Updated 6 months ago
- Generate DBT tests based on sample data☆37Updated 11 months ago
- Code snippets for Data Engineering Design Patterns book☆53Updated 3 weeks ago
- Code for blog at: https://www.startdataengineering.com/post/docker-for-de/☆32Updated 9 months ago
- Containerized end-to-end analytics of Spotify data using Python, dbt, Postgres, and Metabase☆124Updated 2 years ago
- An example dbt project using AutomateDV to create a Data Vault 2.0 Data Warehouse based on the Snowflake TPC-H dataset.☆45Updated 10 months ago
- Code for dbt tutorial☆149Updated 8 months ago
- Cloned by the `dbt init` task☆60Updated 9 months ago
- A portable Datamart and Business Intelligence suite built with Docker, Airflow, dbt, PostgreSQL and Superset☆38Updated 2 months ago
- Full stack data engineering tools and infrastructure set-up☆48Updated 3 years ago
- Sample project to demonstrate data engineering best practices☆175Updated 11 months ago
- A self-contained, ready to run Airflow ELT project. Can be run locally or within codespaces.☆62Updated last year
- Execution of DBT models using Apache Airflow through Docker Compose☆113Updated 2 years ago
- ☆33Updated this week