moritzkoerber / covid-19-data-engineering-pipeline
A Covid-19 data pipeline on AWS featuring PySpark/Glue, Docker, Great Expectations, Airflow, and Redshift, templated in CloudFormation and CDK, deployable via Github Actions.
☆23Updated last year
Alternatives and similar repositories for covid-19-data-engineering-pipeline:
Users that are interested in covid-19-data-engineering-pipeline are comparing it to the libraries listed below
- End to end data engineering project☆54Updated 2 years ago
- Code for my "Efficient Data Processing in SQL" book.☆56Updated 8 months ago
- Open Data Stack Projects: Examples of End to End Data Engineering Projects☆82Updated last year
- build dw with dbt☆44Updated 6 months ago
- A modern ELT demo using airbyte, dbt, snowflake and dagster☆27Updated 2 years ago
- Code for blog at: https://www.startdataengineering.com/post/docker-for-de/☆36Updated 11 months ago
- ☆78Updated 6 months ago
- Simple stream processing pipeline☆101Updated 10 months ago
- Full stack data engineering tools and infrastructure set-up☆52Updated 4 years ago
- Generate DBT tests based on sample data☆36Updated last year
- Template for Data Engineering and Data Pipeline projects☆109Updated 2 years ago
- Repo for orienting dbt users to the Dagster asset framework☆54Updated 2 years ago
- A portable Datamart and Business Intelligence suite built with Docker, sqlmesh + dbtcore, DuckDB and Superset☆49Updated 5 months ago
- Cost Efficient Data Pipelines with DuckDB☆52Updated 8 months ago
- ☆36Updated last month
- dbt Project for Rapid Onboarding instructors to use in instruction and learners to reference throughout the course.☆26Updated this week
- A "modern" Strava data pipeline fueled by dlt, duckdb, dbt, and evidence.dev☆32Updated 3 months ago
- An example dbt project using AutomateDV to create a Data Vault 2.0 Data Warehouse based on the Snowflake TPC-H dataset.☆46Updated last year
- Code to demonstrate data engineering metadata & logging best practices☆16Updated last year
- Code for "Advanced data transformations in SQL" free live workshop☆78Updated 6 months ago
- ☆114Updated 9 months ago
- Example repo to create end to end tests for data pipeline.☆23Updated 10 months ago
- ☆17Updated 8 months ago
- Code for dbt tutorial☆156Updated 10 months ago
- A demonstration of an ELT (Extract, Load, Transform) pipeline☆29Updated last year
- A simple and easy to use Data Quality (DQ) tool built with Python.☆50Updated last year
- ☆21Updated last year
- Some example projects for Data Engineers to build, end-to-end.☆28Updated last year
- A portable Datamart and Business Intelligence suite built with Docker, Airflow, dbt, PostgreSQL and Superset☆41Updated 5 months ago
- velib-v2: An ETL pipeline that employs batch and streaming jobs using Spark, Kafka, Airflow, and other tools, all orchestrated with Docke…☆18Updated 7 months ago