karenbajador / pyspark_greatexpectations

Related projects ⓘ

Alternatives and complementary repositories for pyspark_greatexpectations

• vim89 / datapipelines-essentials-python
  Simplified ETL process in Hadoop using Apache Spark. Has complete ETL pipeline for datalake. SparkSession extensions, DataFrame validatio…
  ☆53Updated last year
• hnawaz007 / dbt-dw
  build dw with dbt
  ☆29Updated 2 weeks ago
• justhackit / spark-utils
  ☆10Updated 2 years ago
• ismaildawoodjee / GreatEx
  A project for exploring how Great Expectations can be used to ensure data quality and validate batches within a data pipeline defined in …
  ☆21Updated 2 years ago
• cordon-thiago / spark-schema-merge
  Spark app to merge different schemas
  ☆23Updated 3 years ago
• arezamoosavi / AcidOnSpark-ETL
  Delta-Lake, ETL, Spark, Airflow
  ☆44Updated 2 years ago
• Data-Engineer-Camp / modern-elt-demo
  A modern ELT demo using airbyte, dbt, snowflake and dagster
  ☆24Updated last year
• rafaelpierre / pyjaws
  PyJaws: A Pythonic Way to Define Databricks Jobs and Workflows
  ☆41Updated 4 months ago
• Wittline / pyspark-on-aws-emr
  The goal of this project is to offer an AWS EMR template using Spot Fleet and On-Demand Instances that you can use quickly. Just focus on…
  ☆25Updated 2 years ago
• josephmachado / cost_effective_data_pipelines
  Cost Efficient Data Pipelines with DuckDB
  ☆46Updated 3 months ago
• 1997mahadi / dbt-dlt-ingestion-pipeline
  ☆15Updated 3 months ago
• saikrishnapujari / Spark-Nested-Data-Parser
  Nested Data (JSON/AVRO/XML) Parsing and Flattening in Spark
  ☆15Updated 9 months ago
• dqops / dqo
  Data Quality and Observability platform for the whole data lifecycle, from profiling new data sources to full automation with Data Observ…
  ☆111Updated this week
• mikulskibartosz / check-engine
  Data validation library for PySpark 3.0.0
  ☆34Updated last year
• ssp-data / data-engineering-devops
  Full stack data engineering tools and infrastructure set-up
  ☆41Updated 3 years ago
• delta-io / delta-examples
  Delta Lake examples
  ☆205Updated last month
• delta-io / delta-docs
  Delta Lake Documentation
  ☆46Updated 4 months ago
• yennanliu / NYC_Taxi_Pipeline
  Design/Implement stream/batch architecture on NYC taxi data | #DE
  ☆26Updated 3 years ago
• jonathanneo / data-aware-orchestration
  Data-aware orchestration with dagster, dbt, and airbyte
  ☆30Updated last year
• Divergent-Insights / dbt-dataquality
  Creates simple data models on Snowflake to report dbt source freshness and tests
  ☆22Updated last year
• Faire / snowflake-migration
  Repository containing various utils related to Snowflake migration at Faire.
  ☆11Updated last year
• dbt-labs / semantic-layer-llm-benchmarking
  ☆29Updated 10 months ago
• jeppe742 / DeltaLakeReader
  Read Delta tables without any Spark
  ☆47Updated 8 months ago
• Data-Engineer-Camp / dbt-dimensional-modelling
  Step-by-step tutorial on building a Kimball dimensional model with dbt
  ☆109Updated 3 months ago
• josephmachado / simple_dbt_project
  Code for dbt tutorial
  ☆143Updated 5 months ago
• Spratiher9 / JumpSpark
  JumpSpark - A modern cookiecutter template for pyspark projects with batteries included.
  ☆10Updated last year
• PacktPublishing / Driving-Data-Quality-with-Data-Contracts
  ☆32Updated 5 months ago
• mrpowers-io / levi
  Delta Lake helper methods. No Spark dependency.
  ☆22Updated 2 months ago