anilkulkarni87 / airflow-dockerLinks
This is my Apache Airflow Local development setup on Windows 10 WSL2/Mac using docker-compose. It will also include some sample DAGs and workflows.
☆31Updated last year
Alternatives and similar repositories for airflow-docker
Users that are interested in airflow-docker are comparing it to the libraries listed below
Sorting:
- Deploying a Machine Learning model streaming application with Apache Kafka☆10Updated 2 years ago
- Built a stream processing data pipeline to get data from disparate systems into a dashboard using Kafka as an intermediary.☆29Updated last year
- Apache Airflow advanced functionalities examples☆19Updated last year
- ☆12Updated 3 years ago
- Delta-Lake, ETL, Spark, Airflow☆47Updated 2 years ago
- ☆37Updated 5 years ago
- To provide a deeper understanding of how the modern, open-source data stack consisting of Iceberg, dbt, Trino, and Hive operates within a…☆35Updated last year
- Sample Airflow DAGs to load data from the CovidTracking API to Snowflake via an AWS S3 intermediary.☆16Updated 4 years ago
- build dw with dbt☆46Updated 8 months ago
- Developed a data pipeline to automate data warehouse ETL by building custom airflow operators that handle the extraction, transformation,…☆90Updated 3 years ago
- ☆12Updated 3 years ago
- Challenge Data Engineer☆25Updated 3 years ago
- A data pipeline moving data from a Relational database system (RDBMS) to a Hadoop file system (HDFS).☆14Updated 4 years ago
- Public source code for the Batch Processing with Apache Beam (Python) online course☆18Updated 4 years ago
- Design/Implement stream/batch architecture on NYC taxi data | #DE☆25Updated 4 years ago
- PipeRider dbt workshop for DataTalksClub DE Zoomcamp☆18Updated last year
- ☆41Updated 11 months ago
- Docker compose and Google Colab demo to build a CDC with Delta Lake☆15Updated 2 years ago
- Demo on how to use Prefect 2 in an ML project☆41Updated 2 years ago
- A Series of Notebooks on how to start with Kafka and Python☆154Updated 3 months ago
- dlt-dagster-demo☆11Updated last year
- Kafka variant of the MLOps Level 1 stack☆24Updated 3 years ago
- Big Data Demystified meetup and blog examples☆31Updated 10 months ago
- A portable Datamart and Business Intelligence suite built with Docker, Airflow, dbt, PostgreSQL and Superset☆43Updated 7 months ago
- Project for real-time anomaly detection using Kafka and python☆58Updated 2 years ago
- Simple alert system implemented in Kafka and Python☆95Updated 7 years ago
- Docker environment that spins up MongoDB replica set, Spark, and Jupyter Lab. Example code uses PySpark and the MongoDB Spark Connector.☆40Updated 2 years ago
- A Postgres data warehouse for processing synthetic data using IAC principles☆18Updated 2 years ago
- A repository of sample code to show data quality checking best practices using Airflow.☆77Updated 2 years ago
- Simplify Big Data Analytics with Amazon EMR, published by Packt☆13Updated 2 years ago