Anant / example-airflow-and-spark

Related projects ⓘ

Alternatives and complementary repositories for example-airflow-and-spark

• rafaelvp-db / databricks-end-to-end-streaming
  End-to-end Kafka Streaming Examples on Databricks with Evolving Avro Schemas.
  ☆9Updated 9 months ago
• arezamoosavi / AcidOnSpark-ETL
  Delta-Lake, ETL, Spark, Airflow
  ☆44Updated 2 years ago
• dsynkov / spark-livy-on-airflow-workspace
  A workspace to experiment with Apache Spark, Livy, and Airflow in a Docker environment.
  ☆39Updated 3 years ago
• cordon-thiago / airflow-spark
  Docker with Airflow and Spark standalone cluster
  ☆246Updated last year
• raveendratal / ravi_azureadbadf
  Ravi Azure ADB ADF Repository
  ☆64Updated 7 months ago
• derar-alhussein / Databricks-Certified-Data-Engineer-Professional
  The resources of the preparation course for Databricks Data Engineer Professional certification exam
  ☆86Updated last month
• pyjaime / docker-airflow-spark
  Docker with Airflow + Postgres + Spark cluster + JDK (spark-submit support) + Jupyter Notebooks
  ☆21Updated 2 years ago
• nburkett / ShowPulse_Ticketmaster
  A data pipeline with Kafka, Spark Streaming, dbt, Docker, Airflow, and GCP!
  ☆11Updated last year
• LearningJournal / Spark-Streaming-In-Python
  Apache Spark 3 - Structured Streaming Course Material
  ☆119Updated last year
• martandsingh / ApacheSpark
  This repository will help you to learn about databricks concept with the help of examples. It will include all the important topics which…
  ☆92Updated 3 months ago
• luanmorenomaciel / de-apache-spark
  Data Engineering com Apache Spark
  ☆43Updated 3 years ago
• ajupton / big-data-engineering-project
  Big Data Engineering practice project, including ETL with Airflow and Spark using AWS S3 and EMR
  ☆80Updated 5 years ago
• aws-dojo / analytics
  code snippet for analytics sessions
  ☆33Updated 2 years ago
• itversity / data-engineering-spark
  ☆86Updated 2 years ago
• Realsid / databricks-spark-certification
  Guide for databricks spark certification
  ☆58Updated 3 years ago
• itversity / spark-sql-and-pyspark-using-python3
  Repository related to Spark SQL and Pyspark using Python3
  ☆36Updated 2 years ago
• SebasMBK / SunglassesHubETL
  This project is for demonstrating knowledge of Data Engineering tools and concepts and also learning in the process
  ☆45Updated last year
• josephmachado / spark_submit_airflow
  Simple repo to demonstrate how to submit a spark job to EMR from Airflow
  ☆32Updated 4 years ago
• guidok91 / spark-movies-etl
  Spark data pipeline that processes movie ratings data.
  ☆27Updated last week
• mrn-aglic / pyspark-playground
  ☆69Updated 5 months ago
• techops-recsys-lateral-hiring / dataengineer-transformations-python
  ☆44Updated 2 weeks ago
• vim89 / datapipelines-essentials-python
  Simplified ETL process in Hadoop using Apache Spark. Has complete ETL pipeline for datalake. SparkSession extensions, DataFrame validatio…
  ☆53Updated last year
• Amaguk2023 / Pyspark_Spotify_ETL
  Pyspark Spotify ETL
  ☆17Updated 3 years ago
• jonathanneo / databricks-unit-testing
  Unit testing using databricks connect
  ☆30Updated 3 years ago
• YFChiu / Resources--Databricks-Certified-Associate-Developer-for-Apache-Spark-3.0
  (Python, PySpark)
  ☆11Updated 4 years ago
• shravan-kuchkula / udacity-data-eng-proj4
  Developed an ETL pipeline for a Data Lake that extracts data from S3, processes the data using Spark, and loads the data back into S3 as …
  ☆16Updated 5 years ago
• josephmachado / bitcoinMonitor
  Near real time ETL to populate a dashboard.
  ☆70Updated 5 months ago
• garystafford / tickit-data-lake-demo
  Resources for video demonstrations and blog posts related to DataOps on AWS
  ☆170Updated 2 years ago