dsynkov / spark-livy-on-airflow-workspace

Related projects ⓘ

Alternatives and complementary repositories for spark-livy-on-airflow-workspace

• cordon-thiago / airflow-spark
  Docker with Airflow and Spark standalone cluster
  ☆245Updated last year
• josephmachado / spark_submit_airflow
  Simple repo to demonstrate how to submit a spark job to EMR from Airflow
  ☆32Updated 4 years ago
• LearningJournal / Spark-Streaming-In-Python
  Apache Spark 3 - Structured Streaming Course Material
  ☆119Updated last year
• sdesilva26 / docker-spark
  Tutorial for setting up a Spark cluster running inside of Docker containers located on different machines
  ☆123Updated 2 years ago
• yTek01 / docker-spark-airflow
  ☆32Updated last year
• josephmachado / beginner_de_project_stream
  Simple stream processing pipeline
  ☆92Updated 5 months ago
• arezamoosavi / AcidOnSpark-ETL
  Delta-Lake, ETL, Spark, Airflow
  ☆44Updated 2 years ago
• vim89 / datapipelines-essentials-python
  Simplified ETL process in Hadoop using Apache Spark. Has complete ETL pipeline for datalake. SparkSession extensions, DataFrame validatio…
  ☆53Updated last year
• konosp / dbt-airflow-docker-compose
  Execution of DBT models using Apache Airflow through Docker Compose
  ☆113Updated last year
• akashmehta10 / profiling_pyspark
  ☆25Updated last year
• mrn-aglic / spark-standalone-cluster
  This repo contains a spark standalone cluster on docker for anyone who wants to play with PySpark by submitting their applications.
  ☆23Updated last year
• delta-io / delta-examples
  Delta Lake examples
  ☆207Updated last month
• garystafford / tickit-data-lake-demo
  Resources for video demonstrations and blog posts related to DataOps on AWS
  ☆170Updated 2 years ago
• Realsid / databricks-spark-certification
  Guide for databricks spark certification
  ☆58Updated 3 years ago
• josephmachado / online_store
  End to end data engineering project
  ☆51Updated 2 years ago
• NAVEENKUMARMURUGAN / Pyspark-ETL-Framework
  ☆14Updated 5 years ago
• guidok91 / spark-movies-etl
  Spark data pipeline that processes movie ratings data.
  ☆27Updated this week
• greatexpectationslabs / ge_tutorials
  Learn how to add data validation and documentation to a data pipeline built with dbt and Airflow.
  ☆166Updated last year
• ovimihai / airflow-cert-dag-authoring
  Course notes for the Astronomer Certification DAG Authoring for Apache Airflow
  ☆51Updated 8 months ago
• cluster-apps-on-docker / spark-standalone-cluster-on-docker
  Learn Apache Spark in Scala, Python (PySpark) and R (SparkR) by building your own cluster with a JupyterLab interface on Docker.
  ☆462Updated last year
• derar-alhussein / Databricks-Certified-Data-Engineer-Professional
  The resources of the preparation course for Databricks Data Engineer Professional certification exam
  ☆86Updated last month
• xnuinside / airflow_in_docker_compose
  Apache Airflow in Docker Compose (for both versions 1.10.* and 2.*)
  ☆184Updated 11 months ago
• garystafford / streaming-sales-generator
  Streaming Synthetic Sales Data Generator: Streaming sales data generator for Apache Kafka, written in Python
  ☆44Updated last year
• subhamkharwal / ease-with-apache-spark
  Series follows learning from Apache Spark (PySpark) with quick tips and workaround for daily problems in hand
  ☆42Updated last year
• josephmachado / socialetl
  Project for "Data pipeline design patterns" blog.
  ☆41Updated 3 months ago
• marclamberti / airflow-eks-helm-chart
  Airflow helm chart for AWS EKS
  ☆18Updated 3 years ago
• mahdyne / pyspark-tut
  ☆23Updated 3 years ago
• astronomer / airflow-data-quality-demo
  A repository of sample code to show data quality checking best practices using Airflow.
  ☆72Updated last year
• ananthdurai / airflow-training
  Airflow training for the crunch conf
  ☆105Updated 6 years ago