emmc15 / pyspark-testing-envLinks
Example Repo to have full end to end pyspark testing via docker-compose
☆31Updated 2 years ago
Alternatives and similar repositories for pyspark-testing-env
Users that are interested in pyspark-testing-env are comparing it to the libraries listed below
Sorting:
- Delta Lake helper methods in PySpark☆325Updated last year
- Quickstart for any service☆161Updated this week
- A Python package that creates fine-grained dbt tasks on Apache Airflow☆71Updated last week
- PySpark test helper methods with beautiful error messages☆714Updated last week
- The Lakehouse Engine is a configuration driven Spark framework, written in Python, serving as a scalable and distributed engine for sever…☆266Updated 2 weeks ago
- A lightweight Python-based tool for extracting and analyzing data column lineage for dbt projects☆180Updated 6 months ago
- Modern serverless lakehouse implementing HOOK methodology, Unified Star Schema (USS), and Analytical Data Storage System (ADSS) principle…☆115Updated 5 months ago
- Slow & local data allows you to move fast and deliver business value for the 99.9% of the data challenges.☆294Updated 5 months ago
- CLI tool for dbt users to simplify creation of staging models (yml and sql) files☆266Updated this week
- A CLI tool to streamline getting started with Apache Airflow™ and managing multiple Airflow projects☆220Updated 4 months ago
- ☆42Updated 4 years ago
- Code for "Efficient Data Processing in Spark" Course☆339Updated 4 months ago
- Dagster Labs' open-source data platform, built with Dagster.☆394Updated this week
- A portable Datamart and Business Intelligence suite built with Docker, Dagster, dbt, DuckDB and Superset☆245Updated 7 months ago
- Data pipeline with dbt, Airflow, Great Expectations☆163Updated 4 years ago
- A collection of Airflow operators, hooks, and utilities to elevate dbt to a first-class citizen of Airflow.☆208Updated last month
- Astro SDK allows rapid and clean development of {Extract, Load, Transform} workflows using Python and SQL, powered by Apache Airflow.☆375Updated 4 months ago
- Pythonic Programming Framework to orchestrate jobs in Databricks Workflow☆219Updated 2 months ago
- Sample Data Lakehouse deployed in Docker containers using Apache Iceberg, Minio, Trino and a Hive Metastore. Can be used for local testin…☆74Updated 2 years ago
- This repository has moved into https://github.com/dbt-labs/dbt-adapters☆251Updated 7 months ago
- Linter for dbt metadata☆175Updated this week
- Adds autocompletion to the dbt CLI☆131Updated 7 months ago
- This dbt package contains macros to support unit testing that can be (re)used across dbt projects.☆443Updated 7 months ago
- A Python Library to support running data quality rules while the spark job is running⚡☆188Updated this week
- Showcase of advanced use cases relating to CI in dbt☆89Updated this week
- Schema modelling framework for decentralised domain-driven ownership of data.☆259Updated last year
- Learn how to add data validation and documentation to a data pipeline built with dbt and Airflow.☆169Updated last year
- ☆159Updated last month
- A curated list of awesome blogs, videos, tools and resources about Data Contracts☆180Updated last year
- Snowflake Grant Report offers a way of visualizing role hierarchy and rapid diagnosis of as-is permissions, giving customers insight with…☆75Updated 3 years ago