r-kells / prod-airflow
Helping you get Airflow running in production.
☆9Updated 5 years ago
Alternatives and similar repositories for prod-airflow:
Users that are interested in prod-airflow are comparing it to the libraries listed below
- Udacity Data Pipeline Exercises☆15Updated 4 years ago
- ☆18Updated 3 years ago
- Design/Implement stream/batch architecture on NYC taxi data | #DE☆25Updated 3 years ago
- Simplified ETL process in Hadoop using Apache Spark. Has complete ETL pipeline for datalake. SparkSession extensions, DataFrame validatio…☆53Updated last year
- event-triggered plugins for airflow☆21Updated 5 years ago
- PySpark phonetic and string matching algorithms☆39Updated last year
- Just a boilerplate for PySpark and Flask☆35Updated 6 years ago
- An example PySpark project with pytest☆17Updated 7 years ago
- Repository used for Spark Trainings☆53Updated last year
- Project files for the post: Running PySpark Applications on Amazon EMR using Apache Airflow: Using the new Amazon Managed Workflows for A…☆40Updated 2 years ago
- Blog post on ETL pipelines with Airflow☆23Updated 4 years ago
- PyConDE & PyData Berlin 2019 Airflow Workshop: Airflow for machine learning pipelines.☆47Updated last year
- ☆16Updated 7 years ago
- ☆11Updated 3 years ago
- Data models, build data warehouses and data lakes, automate data pipelines, and worked with massive datasets.☆13Updated 5 years ago
- Basic tutorial of using Apache Airflow☆36Updated 6 years ago
- ☆19Updated 4 years ago
- Example script to deploy DAGs to Google Cloud Composer.☆15Updated 2 years ago
- This repository shows a sample example to build, manage and orchestrate Machine Learning workflows using Amazon Sagemaker and Apache Airf…☆136Updated 3 years ago
- Docker compose and Google Colab demo to build a CDC with Delta Lake☆15Updated 2 years ago
- Snowflake Guide: Building a Recommendation Engine Using Snowflake & Amazon SageMaker☆31Updated 3 years ago
- A repo to track data engineering projects☆13Updated 2 years ago
- (project & tutorial) dag pipeline tests + ci/cd setup☆86Updated 4 years ago
- Examples for using Amazon SageMaker components in Kubeflow Pipelines☆22Updated 4 years ago
- Code to build a simple analytics data pipeline with Python☆102Updated 8 years ago
- scaffold of Apache Airflow executing Docker containers☆85Updated 2 years ago
- Example of an ETL Pipeline using Airflow☆34Updated 7 years ago
- Repo for all my code on the articles I post on medium☆107Updated 2 years ago
- Generalized project for running Airflow DAGs, with possibility of skipping tasks already done for some set of input parameters.☆15Updated 2 years ago
- Airflow helm chart for AWS EKS☆18Updated 4 years ago