astronomer / airflow-covid-data
Sample Airflow DAGs to load data from the CovidTracking API to Snowflake via an AWS S3 intermediary.
☆16Updated 4 years ago
Alternatives and similar repositories for airflow-covid-data:
Users that are interested in airflow-covid-data are comparing it to the libraries listed below
- Sample Airflow DAGs☆62Updated 2 years ago
- A repository of sample code to show data quality checking best practices using Airflow.☆74Updated 2 years ago
- Data lake, data warehouse on GCP☆56Updated 3 years ago
- Simplified ETL process in Hadoop using Apache Spark. Has complete ETL pipeline for datalake. SparkSession extensions, DataFrame validatio…☆53Updated last year
- Execution of DBT models using Apache Airflow through Docker Compose☆116Updated 2 years ago
- Code snippets for Data Engineering Design Patterns book☆74Updated last week
- Developed a data pipeline to automate data warehouse ETL by building custom airflow operators that handle the extraction, transformation,…☆90Updated 3 years ago
- Analytics engineering with dbt - projects and developer environment☆17Updated 6 months ago
- Airflow Examples: code samples for Medium articles☆13Updated 4 years ago
- This project helps me to understand the core concepts of Apache Airflow. I have created custom operators to perform tasks such as staging…☆82Updated 5 years ago
- Project files for the post: Running PySpark Applications on Amazon EMR using Apache Airflow: Using the new Amazon Managed Workflows for A…☆40Updated 2 years ago
- Code for dbt tutorial☆153Updated 9 months ago
- Cloned by the `dbt init` task☆61Updated 11 months ago
- Full stack data engineering tools and infrastructure set-up☆50Updated 4 years ago
- A repository of sample code to accompany our blog post on Airflow and dbt.☆170Updated last year
- Source code for the YouTube video, Apache Beam Explained in 12 Minutes☆21Updated 4 years ago
- ☆87Updated 2 years ago
- Slowly Changing Dimension type 2 using Hive query language using exclusive join technique with ORC Hive tables, partitioned and clustered…☆16Updated 5 years ago
- Spark data pipeline that processes movie ratings data.☆28Updated this week
- Streaming Synthetic Sales Data Generator: Streaming sales data generator for Apache Kafka, written in Python☆43Updated 2 years ago
- This project demonstrates how to build and automate an ETL pipeline using DAGs in Airflow and load the transformed data to Bigquery. Ther…☆21Updated this week
- data engineering 100 days 🤖 🧲 🦾 | #DE☆40Updated last year
- Data Engineering with Spark and Delta Lake☆96Updated 2 years ago
- Build & Learn Data Engineering,Machine Learning over Kubernetes. No Shortcut approach.☆57Updated 2 years ago
- Resources for video demonstrations and blog posts related to DataOps on AWS☆172Updated 3 years ago
- Source code for the MC technical blog post "Data Observability in Practice Using SQL"☆36Updated 8 months ago
- Basic tutorial of using Apache Airflow☆36Updated 6 years ago
- Simple stream processing pipeline☆99Updated 9 months ago
- A full data warehouse infrastructure with ETL pipelines running inside docker on Apache Airflow for data orchestration, AWS Redshift for …☆134Updated 4 years ago
- ☆75Updated 5 months ago