Anant / example-airflow-and-spark

Related projects: ⓘ

• itversity / data-engineering-spark
  ☆84Updated 2 years ago
• rafaelvp-db / databricks-end-to-end-streaming
  End-to-end Kafka Streaming Examples on Databricks with Evolving Avro Schemas.
  ☆9Updated 6 months ago
• ajupton / big-data-engineering-project
  Big Data Engineering practice project, including ETL with Airflow and Spark using AWS S3 and EMR
  ☆79Updated 5 years ago
• LearningJournal / Spark-Streaming-In-Python
  Apache Spark 3 - Structured Streaming Course Material
  ☆120Updated last year
• luanmorenomaciel / data-engineering-databricks
  ☆20Updated this week
• raveendratal / ravi_azureadbadf
  Ravi Azure ADB ADF Repository
  ☆64Updated 4 months ago
• luanmorenomaciel / de-apache-spark
  Data Engineering com Apache Spark
  ☆43Updated 3 years ago
• yTek01 / docker-spark-airflow
  ☆30Updated last year
• cordon-thiago / airflow-spark
  Docker with Airflow and Spark standalone cluster
  ☆239Updated last year
• garystafford / tickit-data-lake-demo
  Resources for video demonstrations and blog posts related to DataOps on AWS
  ☆166Updated 2 years ago
• Realsid / databricks-spark-certification
  Guide for databricks spark certification
  ☆57Updated 3 years ago
• supratim94336 / DataEngineeringCapstoneProject
  😈Complete End to End ETL Pipeline with Spark, Airflow, & AWS
  ☆39Updated 5 years ago
• derar-alhussein / Databricks-Certified-Data-Engineer-Professional
  The resources of the preparation course for Databricks Data Engineer Professional certification exam
  ☆73Updated 9 months ago
• josephmachado / bitcoinMonitor
  Near real time ETL to populate a dashboard.
  ☆69Updated 3 months ago
• SebasMBK / SunglassesHubETL
  This project is for demonstrating knowledge of Data Engineering tools and concepts and also learning in the process
  ☆44Updated last year
• arezamoosavi / AcidOnSpark-ETL
  Delta-Lake, ETL, Spark, Airflow
  ☆42Updated last year
• vivek-bombatkar / Spark-with-Python---My-learning-notes-
  ETL pipeline using pyspark (Spark - Python)
  ☆106Updated 4 years ago
• dsynkov / spark-livy-on-airflow-workspace
  A workspace to experiment with Apache Spark, Livy, and Airflow in a Docker environment.
  ☆39Updated 3 years ago
• alanchn31 / Movalytics-Data-Warehouse
  Data pipeline performing ETL to AWS Redshift using Spark, orchestrated with Apache Airflow
  ☆127Updated 4 years ago
• zhujun98 / data-engineering
  Spark, Airflow, Kafka
  ☆27Updated last year
• arverma / TowardsDataEngineering
  This repo contains commands that data engineers use in day to day work.
  ☆58Updated last year
• mrugankray / Big-Data-Cluster
  The goal of this project is to build a docker cluster that gives access to Hadoop, HDFS, Hive, PySpark, Sqoop, Airflow, Kafka, Flume, Pos…
  ☆48Updated last year
• afaqueahmad7117 / spark-experiments
  Ultimate guide for mastering Spark Performance Tuning and Optimization concepts and for preparing for Data Engineering interviews
  ☆57Updated 4 months ago
• Wittline / uber-expenses-tracking
  The goal of this project is to track the expenses of Uber Rides and Uber Eats through data Engineering processes using technologies such …
  ☆111Updated 2 years ago
• shravan-kuchkula / udacity-data-eng-proj4
  Developed an ETL pipeline for a Data Lake that extracts data from S3, processes the data using Spark, and loads the data back into S3 as …
  ☆16Updated 4 years ago
• jacob1421 / RustCheatersDataPipeline
  Data pipeline that scrapes Rust cheater Steam profiles
  ☆50Updated 2 years ago
• databricks-academy / just-enough-python-for-spark
  ☆39Updated this week
• itversity / data-engineering-on-gcp
  Data Engineering on GCP
  ☆29Updated last year
• martandsingh / ApacheSpark
  This repository will help you to learn about databricks concept with the help of examples. It will include all the important topics which…
  ☆91Updated last month
• jonathanneo / databricks-unit-testing
  Unit testing using databricks connect
  ☆29Updated 2 years ago