astronomer / airflow-guide-passing-data-between-tasks
☆10Updated 4 years ago
Alternatives and similar repositories for airflow-guide-passing-data-between-tasks:
Users that are interested in airflow-guide-passing-data-between-tasks are comparing it to the libraries listed below
- Sample Airflow DAGs to load data from the CovidTracking API to Snowflake via an AWS S3 intermediary.☆16Updated 4 years ago
- The goal of this project is to offer an AWS EMR template using Spot Fleet and On-Demand Instances that you can use quickly. Just focus on…☆27Updated 2 years ago
- Multi-docker container data science / engineering playground (w/ Kafka, Airflow, MLFlow, Tensorflow-Keras / SKLearn) for simulating a mic…☆11Updated 2 years ago
- Simplified ETL process in Hadoop using Apache Spark. Has complete ETL pipeline for datalake. SparkSession extensions, DataFrame validatio…☆54Updated 2 years ago
- Databricks Certified Associate Spark Developer preparation toolkit to setup single node Standalone Spark Cluster along with material in t…☆30Updated last year
- A tutorial that helps Big Data Engineers ramp up faster by getting familiar with PySpark dataframes and functions. It also covers topics …☆20Updated 3 years ago
- Repo for CDC with debezium blog post☆28Updated 7 months ago
- Deploy A/B testing infrastructure in a containerized microservice architecture for Machine Learning applications.☆40Updated 3 months ago
- ☆40Updated 10 months ago
- ☆17Updated 8 months ago
- Build an scikit-learn model to predict churn using customer telco data.☆16Updated 5 months ago
- data engineering 100 days 🤖 🧲 🦾 | #DE☆40Updated last year
- Full stack data engineering tools and infrastructure set-up☆52Updated 4 years ago
- Powerful rapid automatic EDA and feature engineering library with a very easy to use API 🌟☆53Updated 3 years ago
- Deploying a Machine Learning model streaming application with Apache Kafka☆10Updated 2 years ago
- This is my Apache Airflow Local development setup on Windows 10 WSL2/Mac using docker-compose. It will also include some sample DAGs and …☆29Updated last year
- ☆21Updated 2 years ago
- Maternal Health Risk prediction MLOps pipeline☆43Updated 2 years ago
- Blog post on ETL pipelines with Airflow☆23Updated 4 years ago
- A real-time streaming ETL pipeline for streaming and performing sentiment analysis on Twitter data using Apache Kafka, Apache Spark and D…☆30Updated 4 years ago
- ☆12Updated 3 years ago
- Code for my "Efficient Data Processing in SQL" book.☆56Updated 9 months ago
- A Series of Notebooks on how to start with Kafka and Python☆154Updated 2 months ago
- Build a semantic search application with deep learning models.☆14Updated 5 months ago
- ☆18Updated 3 years ago
- Data Engineering Capstone☆17Updated 5 years ago
- PySpark Cheatsheet☆36Updated 2 years ago
- Project for real-time anomaly detection using Kafka and python☆58Updated 2 years ago
- reating a modern data pipeline using a combination of Terraform, AWS Lambda and S3, Snowflake, DBT, Mage AI, and Dash.☆14Updated last year
- pyspark dataframe made easy☆16Updated 3 years ago