maxcotec / Apache-Airflow
Code base for airflow training series Getting easy with Apache Airflow
☆39Updated last year
Alternatives and similar repositories for Apache-Airflow:
Users that are interested in Apache-Airflow are comparing it to the libraries listed below
- Data pipeline performing ETL to AWS Redshift using Spark, orchestrated with Apache Airflow☆137Updated 4 years ago
- Realtime Data Engineering Project☆27Updated last month
- Ravi Azure ADB ADF Repository☆66Updated 3 weeks ago
- Developed an ETL pipeline for a Data Lake that extracts data from S3, processes the data using Spark, and loads the data back into S3 as …☆16Updated 5 years ago
- Simple stream processing pipeline☆98Updated 8 months ago
- Simple ETL pipeline using Python☆25Updated last year
- ☆63Updated this week
- Classwork projects and home works done through Udacity data engineering nano degree☆74Updated last year
- Repository related to Spark SQL and Pyspark using Python3☆37Updated 2 years ago
- 😈Complete End to End ETL Pipeline with Spark, Airflow, & AWS☆43Updated 5 years ago
- ☆87Updated 2 years ago
- Project for "Data pipeline design patterns" blog.☆43Updated 6 months ago
- This project focuses on building a robust data pipeline using Apache Airflow to automate the ingestion of weather data from the OpenWeath…☆21Updated last year
- Ultimate guide for mastering Spark Performance Tuning and Optimization concepts and for preparing for Data Engineering interviews☆106Updated 9 months ago
- End to end data engineering project☆53Updated 2 years ago
- ☆44Updated last year
- Recohut - Learn data engineering, data science☆96Updated last year
- Python and AirFlow - Data Pipeline Orchestration☆17Updated last year
- Writes the CSV file to Postgres, read table and modify it. Write more tables to Postgres with Airflow.☆36Updated last year
- Simplified ETL process in Hadoop using Apache Spark. Has complete ETL pipeline for datalake. SparkSession extensions, DataFrame validatio…☆53Updated last year
- PySpark functions and utilities with examples. Assists ETL process of data modeling☆102Updated 4 years ago
- Course notes for the Astronomer Certification DAG Authoring for Apache Airflow☆52Updated 11 months ago
- YouTube tutorial project☆98Updated last year
- ☆119Updated last week
- Data Engineering with Databricks Cookbook, published by Packt☆67Updated 8 months ago
- This repository will help you to learn about databricks concept with the help of examples. It will include all the important topics which…☆95Updated 6 months ago
- Big Data Engineering practice project, including ETL with Airflow and Spark using AWS S3 and EMR☆80Updated 5 years ago
- An End-to-End ETL data pipeline that leverages pyspark parallel processing to process about 25 million rows of data coming from a SaaS ap…☆25Updated 2 years ago
- Resources for video demonstrations and blog posts related to DataOps on AWS☆172Updated 3 years ago
- ☆16Updated last year