Alternatives and similar repositories for DataEngineerChallenge

Users that are interested in DataEngineerChallenge are comparing it to the libraries listed below

• holdenk / spark-upgrade
  Magic to help Spark pipelines upgrade
  ☆35Updated 8 months ago
• akashmehta10 / cdc_pyspark_hive
  ☆23Updated 2 years ago
• Data-Engineer-Camp / dbt-dimensional-modelling
  Step-by-step tutorial on building a Kimball dimensional model with dbt
  ☆142Updated 11 months ago
• siddd88 / gcp-data-engineering
  Batch Processing , orchestration using Apache Airflow and Google Workflows, spark structured Streaming and a lot more
  ☆18Updated 3 years ago
• dimajix / spark-training
  Repository used for Spark Trainings
  ☆53Updated 2 years ago
• josephmachado / spark_submit_airflow
  Simple repo to demonstrate how to submit a spark job to EMR from Airflow
  ☆33Updated 4 years ago
• sodadata / soda-spark
  Soda Spark is a PySpark library that helps you with testing your data in Spark Dataframes
  ☆64Updated 3 years ago
• karenbajador / pyspark_greatexpectations
  ☆12Updated 3 years ago
• mikulskibartosz / check-engine
  Data validation library for PySpark 3.0.0
  ☆33Updated 2 years ago
• rockthejvm / spark-performance-tuning
  The official repository for the Rock the JVM Spark Optimization 2 course
  ☆40Updated last year
• vivek-bombatkar / MyLearningNotes
  Because its never late to start taking notes and 'public' it...
  ☆59Updated 3 weeks ago
• holdenk / spark-flowchart
  Flowchart for debugging Spark applications
  ☆105Updated 9 months ago
• yennanliu / DE-100-days
  data engineering 100 days 🤖 🧲 🦾 | #DE
  ☆39Updated last year
• josephmachado / simple_dbt_project
  Code for dbt tutorial
  ☆156Updated 3 weeks ago
• rockthejvm / spark-optimization
  The official repository for the Rock the JVM Spark Optimization with Scala course
  ☆58Updated last year
• alvinj / ScalaCookbook2Examples
  Source code examples for the Second Edition of the Scala Cookbook
  ☆47Updated 2 years ago
• MrPowers / ceja
  PySpark phonetic and string matching algorithms
  ☆39Updated last year
• hyunjoonbok / PySpark
  PySpark functions and utilities with examples. Assists ETL process of data modeling
  ☆103Updated 4 years ago
• arezamoosavi / AcidOnSpark-ETL
  Delta-Lake, ETL, Spark, Airflow
  ☆47Updated 2 years ago
• shravan-kuchkula / udacity-data-eng-proj-1
  Developed a data pipeline to automate data warehouse ETL by building custom airflow operators that handle the extraction, transformation,…
  ☆90Updated 3 years ago
• vim89 / datapipelines-essentials-python
  Simplified ETL process in Hadoop using Apache Spark. Has complete ETL pipeline for datalake. SparkSession extensions, DataFrame validatio…
  ☆55Updated 2 years ago
• mrpowers-io / jodie
  Delta lake and filesystem helper methods
  ☆51Updated last year
• josephmachado / e2e_datapipeline_test
  Example repo to create end to end tests for data pipeline.
  ☆25Updated last year
• samelamin / spark-bigquery
  Google BigQuery support for Spark, Structured Streaming, SQL, and DataFrames with easy Databricks integration.
  ☆70Updated 2 years ago
• chandulal / airflow-testing
  Airflow Unit Tests and Integration Tests
  ☆259Updated 2 years ago
• guidok91 / spark-movies-etl
  Spark data pipeline that processes movie ratings data.
  ☆28Updated last week
• pran4ajith / spark-twitter-streaming
  A real-time streaming ETL pipeline for streaming and performing sentiment analysis on Twitter data using Apache Kafka, Apache Spark and D…
  ☆30Updated 4 years ago
• awesome-spark / learn-by-examples
  Real-world Spark pipelines examples
  ☆83Updated 7 years ago
• marcosmarxm / airflow-testing-ci-workflow
  (project & tutorial) dag pipeline tests + ci/cd setup
  ☆88Updated 4 years ago
• actionml / docker-spark
  Apache Spark docker container image (Standalone mode)
  ☆35Updated 4 years ago