dushyantkhosla / airflow4ds
Using Apache Airflow to author, run and monitor complex data pipelines.
☆12Updated 6 years ago
Alternatives and similar repositories for airflow4ds:
Users that are interested in airflow4ds are comparing it to the libraries listed below
- Repository used for Spark Trainings☆53Updated last year
- Airflow training for the crunch conf☆105Updated 6 years ago
- Create HTML profiling reports from Apache Spark DataFrames☆195Updated 5 years ago
- MLFlow Spark Summit 2019 Presentation☆67Updated 5 years ago
- Learn how to add data validation and documentation to a data pipeline built with dbt and Airflow.☆166Updated last year
- ☆84Updated last year
- Use Airflow to move data from multiple MySQL databases to BigQuery☆100Updated 4 years ago
- Airflow ETL for Meetup API☆46Updated 6 years ago
- HandySpark - bringing pandas-like capabilities to Spark dataframes☆192Updated 5 years ago
- Capturing model drift and handling its response - Example webinar☆107Updated 5 years ago
- (project & tutorial) dag pipeline tests + ci/cd setup☆86Updated 4 years ago
- Tutorial like code for how to deploy airflow using docker and how to use the DockerOperator.☆44Updated 5 years ago
- Example of an ETL Pipeline using Airflow☆33Updated 7 years ago
- ☆181Updated 2 years ago
- Example project for the course "Testing & Monitoring Machine Learning Model Deployments"☆134Updated last year
- ☆43Updated last year
- ☆198Updated last year
- A code-based tutorial for production level data streaming with PySpark plus Optimus for data cleaning, Confluent Kafka, & Apache Drill u…☆26Updated 5 years ago
- python automatic data quality check toolkit☆284Updated 4 years ago
- PyConDE & PyData Berlin 2019 Airflow Workshop: Airflow for machine learning pipelines.☆46Updated last year
- A production-grade data pipeline has been designed to automate the parsing of user search patterns to analyze user engagement. Extract d…☆24Updated 3 years ago
- 🐍💨 Airflow tutorial for PyCon 2019☆85Updated 2 years ago
- Developed a data pipeline to automate data warehouse ETL by building custom airflow operators that handle the extraction, transformation,…☆90Updated 3 years ago
- Code to build a simple analytics data pipeline with Python☆102Updated 7 years ago
- Data Exploration in PySpark made easy - Pyspark_dist_explore provides methods to get fast insights in your Spark DataFrames.☆103Updated 5 years ago
- LearningApacheSpark☆246Updated last year
- Because its never late to start taking notes and 'public' it...☆60Updated 3 months ago
- Playing with different packages of the Apache Spark☆28Updated 8 months ago
- This is repository of my YouTube Course on End to End Apache Spark in AIEngineering YouTube Channel☆189Updated 3 years ago
- PySpark Code for Hands-on Learners☆116Updated 5 years ago