nishakanthiA / Airflow-Docker
Orchestration of data science models in Apache Airflow, scale-up with Celery Executor and deploying in multiple Docker containers
☆30Updated 4 years ago
Related projects ⓘ
Alternatives and complementary repositories for Airflow-Docker
- Apache Airflow in Docker Compose (for both versions 1.10.* and 2.*)☆184Updated last year
- Resources for video demonstrations and blog posts related to DataOps on AWS☆170Updated 2 years ago
- A complete development environment setup for working with Airflow☆126Updated last year
- Delta-Lake, ETL, Spark, Airflow☆44Updated 2 years ago
- ☆48Updated 3 years ago
- A workspace to experiment with Apache Spark, Livy, and Airflow in a Docker environment.☆39Updated 3 years ago
- Docker Airflow - Contains a docker compose file for Airflow 2.0☆59Updated 2 years ago
- ☆32Updated last year
- Docker with Airflow and Spark standalone cluster☆246Updated last year
- A Series of Notebooks on how to start with Kafka and Python☆153Updated last year
- Source code of the Apache Airflow Tutorial for Beginners on YouTube Channel Coder2j (https://www.youtube.com/c/coder2j)☆273Updated 8 months ago
- Airflow training for the crunch conf☆105Updated 6 years ago
- (project & tutorial) dag pipeline tests + ci/cd setup☆85Updated 3 years ago
- Learn how to add data validation and documentation to a data pipeline built with dbt and Airflow.☆166Updated last year
- Code for Data Pipelines with Apache Airflow☆720Updated 3 months ago
- ☆180Updated last year
- Execution of DBT models using Apache Airflow through Docker Compose☆113Updated last year
- An example of an ETL pipeline that lays out generic DE processes. This is now out of date but still provides useful information☆27Updated 2 years ago
- ☆86Updated 2 years ago
- This project helps me to understand the core concepts of Apache Airflow. I have created custom operators to perform tasks such as staging…☆74Updated 5 years ago
- Example of an ETL Pipeline using Airflow☆32Updated 7 years ago
- Simple stream processing pipeline☆92Updated 5 months ago
- Airflow Tutorials☆24Updated 3 years ago
- A production-grade data pipeline has been designed to automate the parsing of user search patterns to analyze user engagement. Extract d…☆24Updated 3 years ago
- Building a Modern Data Lake with Minio, Spark, Airflow via Docker.☆15Updated 6 months ago
- Developed a data pipeline to automate data warehouse ETL by building custom airflow operators that handle the extraction, transformation,…☆89Updated 3 years ago
- Data pipeline that scrapes Rust cheater Steam profiles☆51Updated 2 years ago
- A repository of sample code to show data quality checking best practices using Airflow.☆72Updated last year
- Example repo to create end to end tests for data pipeline.☆21Updated 5 months ago
- PyConDE & PyData Berlin 2019 Airflow Workshop: Airflow for machine learning pipelines.☆46Updated last year