cordon-thiago / airflow-spark

Related projects ⓘ

Alternatives and complementary repositories for airflow-spark

• josephmachado / spark_submit_airflow
  Simple repo to demonstrate how to submit a spark job to EMR from Airflow
  ☆32Updated 4 years ago
• cartershanklin / pyspark-cheatsheet
  PySpark Cheat Sheet - example code to help you learn PySpark and develop apps faster
  ☆424Updated last month
• alanchn31 / Movalytics-Data-Warehouse
  Data pipeline performing ETL to AWS Redshift using Spark, orchestrated with Apache Airflow
  ☆133Updated 4 years ago
• josephmachado / data_engineering_project_template
  A template repository to create a data project with IAC, CI/CD, Data migrations, & testing
  ☆241Updated 4 months ago
• dsynkov / spark-livy-on-airflow-workspace
  A workspace to experiment with Apache Spark, Livy, and Airflow in a Docker environment.
  ☆39Updated 3 years ago
• LearningJournal / Spark-Streaming-In-Python
  Apache Spark 3 - Structured Streaming Course Material
  ☆119Updated last year
• arezamoosavi / AcidOnSpark-ETL
  Delta-Lake, ETL, Spark, Airflow
  ☆44Updated 2 years ago
• konosp / dbt-airflow-docker-compose
  Execution of DBT models using Apache Airflow through Docker Compose
  ☆113Updated last year
• cluster-apps-on-docker / spark-standalone-cluster-on-docker
  Learn Apache Spark in Scala, Python (PySpark) and R (SparkR) by building your own cluster with a JupyterLab interface on Docker.
  ☆462Updated last year
• ovimihai / airflow-cert-dag-authoring
  Course notes for the Astronomer Certification DAG Authoring for Apache Airflow
  ☆51Updated 7 months ago
• josephmachado / beginner_de_project_stream
  Simple stream processing pipeline
  ☆92Updated 5 months ago
• tiagotxm / yt-spark-no-kubernetes
  ☆8Updated last month
• josephmachado / online_store
  End to end data engineering project
  ☆51Updated 2 years ago
• RSKriegs / finnhub-streaming-data-pipeline
  Stream processing pipeline from Finnhub websocket using Spark, Kafka, Kubernetes and more
  ☆294Updated 11 months ago
• garystafford / tickit-data-lake-demo
  Resources for video demonstrations and blog posts related to DataOps on AWS
  ☆170Updated 2 years ago
• nordquant / complete-dbt-bootcamp-zero-to-hero
  Supplementary Materials for the The Complete dbt (Data Build Tool) Bootcamp Udemy course
  ☆458Updated last month
• josephmachado / data_engineering_best_practices
  Sample project to demonstrate data engineering best practices
  ☆166Updated 8 months ago
• sdesilva26 / docker-spark
  Tutorial for setting up a Spark cluster running inside of Docker containers located on different machines
  ☆123Updated 2 years ago
• josephmachado / bitcoinMonitor
  Near real time ETL to populate a dashboard.
  ☆70Updated 5 months ago
• Snowflake-Labs / sfguide-data-engineering-with-snowpark-python
  ☆113Updated last month
• josephmachado / simple_dbt_project
  Code for dbt tutorial
  ☆143Updated 5 months ago
• renatootescu / ETL-pipeline
  Educational project on how to build an ETL (Extract, Transform, Load) data pipeline, orchestrated with Airflow.
  ☆297Updated 2 years ago
• ankurchavda / streamify
  A data engineering project with Kafka, Spark Streaming, dbt, Docker, Airflow, Terraform, GCP and much more!
  ☆575Updated 2 years ago
• iam-mhaseeb / Skytrax-Data-Warehouse
  A full data warehouse infrastructure with ETL pipelines running inside docker on Apache Airflow for data orchestration, AWS Redshift for …
  ☆132Updated 4 years ago
• le-oasis / docker-airflow-spark
  Building a Modern Data Lake with Minio, Spark, Airflow via Docker.
  ☆15Updated 6 months ago
• lucaspfigueiredo / elt-pipeline
  Apartments Data Pipeline using Airflow and Spark.
  ☆18Updated 2 years ago
• danielbeach / dataEngineeringTemplate
  Template for Data Engineering and Data Pipeline projects
  ☆104Updated last year
• ajupton / big-data-engineering-project
  Big Data Engineering practice project, including ETL with Airflow and Spark using AWS S3 and EMR
  ☆80Updated 5 years ago
• akashmehta10 / profiling_pyspark
  ☆25Updated last year