ongxuanhong / de02-pyspark-optimization

Related projects ⓘ

Alternatives and complementary repositories for de02-pyspark-optimization

• josephmachado / beginner_de_project_stream
  Simple stream processing pipeline
  ☆92Updated 5 months ago
• developer-advocacy-dremio / quick-guides-from-dremio
  Quick Guides from Dremio on Several topics
  ☆65Updated 3 weeks ago
• dominikhei / Local-Data-LakeHouse
  Sample Data Lakehouse deployed in Docker containers using Apache Iceberg, Minio, Trino and a Hive Metastore. Can be used for local testin…
  ☆56Updated last year
• Nike-Inc / spark-expectations
  A Python Library to support running data quality rules while the spark job is running⚡
  ☆163Updated last week
• delta-io / delta-examples
  Delta Lake examples
  ☆207Updated last month
• 1ambda / lakehouse
  Playground for Lakehouse (Iceberg, Hudi, Spark, Flink, Trino, DBT, Airflow, Kafka, Debezium CDC)
  ☆44Updated last year
• vim89 / datapipelines-essentials-python
  Simplified ETL process in Hadoop using Apache Spark. Has complete ETL pipeline for datalake. SparkSession extensions, DataFrame validatio…
  ☆53Updated last year
• akashmehta10 / profiling_pyspark
  ☆25Updated last year
• delta-io / delta-docs
  Delta Lake Documentation
  ☆46Updated 5 months ago
• josephmachado / simple_dbt_project
  Code for dbt tutorial
  ☆143Updated 5 months ago
• josephmachado / data_engineering_best_practices
  Sample project to demonstrate data engineering best practices
  ☆166Updated 8 months ago
• bitsondatadev / trino-getting-started
  ☆252Updated 3 weeks ago
• ysfesr / Building-Data-LakeHouse
  Creation of a data lakehouse and an ELT pipeline to enable the efficient analysis and use of data
  ☆40Updated 11 months ago
• kaoutaar / end-to-end-etl-pipeline-jcdecaux-API
  velib-v2: An ETL pipeline that employs batch and streaming jobs using Spark, Kafka, Airflow, and other tools, all orchestrated with Docke…
  ☆18Updated 2 months ago
• arezamoosavi / AcidOnSpark-ETL
  Delta-Lake, ETL, Spark, Airflow
  ☆44Updated 2 years ago
• dsynkov / spark-livy-on-airflow-workspace
  A workspace to experiment with Apache Spark, Livy, and Airflow in a Docker environment.
  ☆39Updated 3 years ago
• victorcouste / trino-dbt-demo
  Trino dbt demo project to mix and load BigQuery data with and in a local PostgreSQL database
  ☆70Updated 3 years ago
• TomLous / medium-spark-k8s
  Spark on Kubernetes using Helm
  ☆34Updated 4 years ago
• josephmachado / spark_submit_airflow
  Simple repo to demonstrate how to submit a spark job to EMR from Airflow
  ☆32Updated 4 years ago
• jaceklaskowski / spark-delta-lake-workshop
  Spark and Delta Lake Workshop
  ☆22Updated 2 years ago
• LearningJournal / Spark-Streaming-In-Python
  Apache Spark 3 - Structured Streaming Course Material
  ☆119Updated last year
• naushadh / hive-metastore
  Apache Hive Metastore as a Standalone server in Docker
  ☆67Updated 3 months ago
• MrPowers / mack
  Delta Lake helper methods in PySpark
  ☆304Updated 2 months ago
• newfront / spark-moderndataengineering
  The source code for the book Modern Data Engineering with Apache Spark
  ☆33Updated 2 years ago
• starburstdata / dbt-trino
  The Trino (https://trino.io/) adapter plugin for dbt (https://getdbt.com)
  ☆217Updated this week
• akashmehta10 / cdc_pyspark_hive
  ☆22Updated 2 years ago
• josephmachado / online_store
  End to end data engineering project
  ☆51Updated 2 years ago
• Data-Engineer-Camp / dbt-dimensional-modelling
  Step-by-step tutorial on building a Kimball dimensional model with dbt
  ☆112Updated 4 months ago
• itversity / data-engineering-spark
  ☆86Updated 2 years ago