patelatharva / Data_Pipelines_with_Apache_Airflow

Related projects: ⓘ

• ajupton / big-data-engineering-project
  Big Data Engineering practice project, including ETL with Airflow and Spark using AWS S3 and EMR
  ☆79Updated 5 years ago
• itversity / data-engineering-spark
  ☆84Updated 2 years ago
• alanchn31 / Movalytics-Data-Warehouse
  Data pipeline performing ETL to AWS Redshift using Spark, orchestrated with Apache Airflow
  ☆127Updated 4 years ago
• ddgope / Data-Pipelines-with-Airflow
  This project helps me to understand the core concepts of Apache Airflow. I have created custom operators to perform tasks such as staging…
  ☆69Updated 5 years ago
• LearningJournal / Spark-Streaming-In-Python
  Apache Spark 3 - Structured Streaming Course Material
  ☆120Updated last year
• NAVEENKUMARMURUGAN / Pyspark-ETL-Framework
  ☆14Updated 5 years ago
• josephmachado / spark_submit_airflow
  Simple repo to demonstrate how to submit a spark job to EMR from Airflow
  ☆31Updated 3 years ago
• raveendratal / ravi_azureadbadf
  Ravi Azure ADB ADF Repository
  ☆64Updated 4 months ago
• supratim94336 / DataEngineeringCapstoneProject
  😈Complete End to End ETL Pipeline with Spark, Airflow, & AWS
  ☆39Updated 5 years ago
• LearningJournal / Apache-Spark-and-Databricks-Stream-Processing-in-Lakehouse
  ☆35Updated 8 months ago
• manuel-lang / Data-Engineering-Nanodegree
  Solution to all projects of Udacity's Data Engineering Nanodegree: Data Modeling with Postgres & Cassandra, Data Warehouse with Redshift,…
  ☆56Updated last year
• kroudir / Data-Engineer-Nanodegree-Projects-Udacity
  Projects done in the Data Engineer Nanodegree Program by Udacity.com
  ☆83Updated last year
• vim89 / datapipelines-essentials-python
  Simplified ETL process in Hadoop using Apache Spark. Has complete ETL pipeline for datalake. SparkSession extensions, DataFrame validatio…
  ☆53Updated last year
• siddd88 / gcp-data-engineering
  Batch Processing , orchestration using Apache Airflow and Google Workflows, spark structured Streaming and a lot more
  ☆19Updated 2 years ago
• dgadiraju / retail_db
  ☆51Updated 3 years ago
• dsynkov / spark-livy-on-airflow-workspace
  A workspace to experiment with Apache Spark, Livy, and Airflow in a Docker environment.
  ☆39Updated 3 years ago
• ilya-galperin / SF-EvictionTracker
  Tracking and measuring neighborhood and district-level eviction rates in the city of San Francisco.
  ☆138Updated 4 years ago
• vineeths96 / Data-Engineering-Nanodegree
  ☆83Updated this week
• josephmachado / beginner_de_project_stream
  Simple stream processing pipeline
  ☆89Updated 3 months ago
• dgadiraju / itversity-books
  ☆111Updated 4 years ago
• aravindr18 / RedditR--Insight-Data-Engineering-Project
  RedditR for Content Engagement and Recommendation
  ☆20Updated 6 years ago
• vivek-bombatkar / Spark-with-Python---My-learning-notes-
  ETL pipeline using pyspark (Spark - Python)
  ☆106Updated 4 years ago
• cordon-thiago / airflow-spark
  Docker with Airflow and Spark standalone cluster
  ☆239Updated last year
• nareshk1290 / Udacity-Data-Engineering
  Udacity Data Engineering Nano Degree (DEND)
  ☆183Updated 4 years ago
• akashmehta10 / profiling_pyspark
  ☆25Updated last year
• PacktPublishing / Data-Engineering-with-Apache-Spark-Delta-Lake-and-Lakehouse
  Data Engineering with Spark and Delta Lake
  ☆86Updated last year
• derar-alhussein / Databricks-Certified-Data-Engineer-Professional
  The resources of the preparation course for Databricks Data Engineer Professional certification exam
  ☆73Updated 9 months ago
• rbmayer / Udacity-Data-Engineering-Nanodegree
  Udacity Data Engineering Nanodegree Projects
  ☆11Updated 5 years ago
• josephmachado / online_store
  End to end data engineering project
  ☆49Updated last year
• josephmachado / data_engineering_best_practices
  Sample project to demonstrate data engineering best practices
  ☆156Updated 6 months ago