guidok91 / spark-movies-etl

Related projects ⓘ

Alternatives and complementary repositories for spark-movies-etl

• vim89 / datapipelines-essentials-python
  Simplified ETL process in Hadoop using Apache Spark. Has complete ETL pipeline for datalake. SparkSession extensions, DataFrame validatio…
  ☆53Updated last year
• astronomer / airflow-data-quality-demo
  A repository of sample code to show data quality checking best practices using Airflow.
  ☆72Updated last year
• supratim94336 / DataEngineeringCapstoneProject
  😈Complete End to End ETL Pipeline with Spark, Airflow, & AWS
  ☆43Updated 5 years ago
• hyunjoonbok / PySpark
  PySpark functions and utilities with examples. Assists ETL process of data modeling
  ☆99Updated 3 years ago
• itversity / data-engineering-spark
  ☆86Updated 2 years ago
• bartosz25 / data-engineering-design-patterns-book
  Code snippets for Data Engineering Design Patterns book
  ☆40Updated this week
• sankamuk / PysparkCheatsheet
  PySpark Cheatsheet
  ☆35Updated last year
• josephmachado / online_store
  End to end data engineering project
  ☆51Updated 2 years ago
• delta-io / delta-docs
  Delta Lake Documentation
  ☆46Updated 5 months ago
• josephmachado / analytical_dp_with_sql
  Code for my "Efficient Data Processing in SQL" book.
  ☆50Updated 3 months ago
• josephmachado / spark_submit_airflow
  Simple repo to demonstrate how to submit a spark job to EMR from Airflow
  ☆32Updated 4 years ago
• akashmehta10 / profiling_pyspark
  ☆25Updated last year
• PacktPublishing / Data-Engineering-with-Apache-Spark-Delta-Lake-and-Lakehouse
  Data Engineering with Spark and Delta Lake
  ☆89Updated last year
• josephmachado / beginner_de_project_stream
  Simple stream processing pipeline
  ☆92Updated 5 months ago
• derar-alhussein / Databricks-Certified-Data-Engineer-Professional
  The resources of the preparation course for Databricks Data Engineer Professional certification exam
  ☆86Updated last month
• josephmachado / simple_dbt_project
  Code for dbt tutorial
  ☆143Updated 5 months ago
• danielbeach / dataEngineeringTemplate
  Template for Data Engineering and Data Pipeline projects
  ☆104Updated last year
• kaxil / airflowctl
  A CLI tool to streamline getting started with Apache Airflow™ and managing multiple Airflow projects
  ☆196Updated this week
• alanchn31 / Movalytics-Data-Warehouse
  Data pipeline performing ETL to AWS Redshift using Spark, orchestrated with Apache Airflow
  ☆133Updated 4 years ago
• PacktPublishing / Data-Engineering-with-Databricks-Cookbook
  Data Engineering with Databricks Cookbook, published by Packt
  ☆45Updated 5 months ago
• josephmachado / e2e_datapipeline_test
  Example repo to create end to end tests for data pipeline.
  ☆21Updated 5 months ago
• mehd-io / pyspark-boilerplate-mehdio
  Pyspark boilerplate for running prod ready data pipeline
  ☆28Updated 3 years ago
• jacob1421 / RustCheatersDataPipeline
  Data pipeline that scrapes Rust cheater Steam profiles
  ☆51Updated 2 years ago
• josephmachado / socialetl
  Project for "Data pipeline design patterns" blog.
  ☆41Updated 3 months ago
• arverma / TowardsDataEngineering
  This repo contains commands that data engineers use in day to day work.
  ☆59Updated last year
• astronomer / airflow-testing-guide
  ☆20Updated 3 years ago
• dsynkov / spark-livy-on-airflow-workspace
  A workspace to experiment with Apache Spark, Livy, and Airflow in a Docker environment.
  ☆39Updated 3 years ago
• ddgope / Data-Pipelines-with-Airflow
  This project helps me to understand the core concepts of Apache Airflow. I have created custom operators to perform tasks such as staging…
  ☆74Updated 5 years ago
• kislerdm / data-engineering-interviews
  Data engineering interviews Q&A for data community by data community
  ☆61Updated 4 years ago
• iam-mhaseeb / Skytrax-Data-Warehouse
  A full data warehouse infrastructure with ETL pipelines running inside docker on Apache Airflow for data orchestration, AWS Redshift for …
  ☆132Updated 4 years ago