godatadriven / pydantic-spark

Related projects ⓘ

Alternatives and complementary repositories for pydantic-spark

• mitchelllisle / sparkdantic
  ✨ A Pydantic to PySpark schema library
  ☆56Updated this week
• benchsci / tinsel
  PySpark schema generator
  ☆38Updated last year
• Nike-Inc / brickflow
  Pythonic Programming Framework to orchestrate jobs in Databricks Workflow
  ☆189Updated last week
• mrpowers-io / levi
  Delta Lake helper methods. No Spark dependency.
  ☆22Updated 2 months ago
• mrpowers-io / jodie
  Delta lake and filesystem helper methods
  ☆49Updated 8 months ago
• coiled / dask-snowflake
  Dask integration for Snowflake
  ☆30Updated last week
• canimus / cuallee
  Possibly the fastest DataFrame-agnostic quality check library in town.
  ☆174Updated this week
• rafaelpierre / pyjaws
  PyJaws: A Pythonic Way to Define Databricks Jobs and Workflows
  ☆41Updated 4 months ago
• delta-incubator / delta-lake-definitive-guide
  ☆13Updated 3 weeks ago
• Nike-Inc / spark-expectations
  A Python Library to support running data quality rules while the spark job is running⚡
  ☆163Updated last week
• delta-io / delta-docs
  Delta Lake Documentation
  ☆46Updated 5 months ago
• jeppe742 / DeltaLakeReader
  Read Delta tables without any Spark
  ☆47Updated 8 months ago
• sodadata / soda-spark
  Soda Spark is a PySpark library that helps you with testing your data in Spark Dataframes
  ☆63Updated 2 years ago
• malexer / pytest-spark
  pytest plugin to run the tests with support of pyspark
  ☆85Updated 8 months ago
• Spratiher9 / JumpSpark
  JumpSpark - A modern cookiecutter template for pyspark projects with batteries included.
  ☆10Updated last year
• delta-io / delta-examples
  Delta Lake examples
  ☆207Updated last month
• MrPowers / farsante
  Fake Pandas / PySpark DataFrame creator
  ☆42Updated 8 months ago
• castor-team / airflow-castor
  ☆45Updated 3 years ago
• gmyrianthous / dbt-airflow
  A Python package that creates fine-grained dbt tasks on Apache Airflow
  ☆62Updated last month
• SemyonSinchenko / flake8-pyspark-with-column
  A flake8 plugin that detects of usage withColumn in a loop or inside reduce
  ☆21Updated last month
• Snowflake-Labs / snowflake-vcrpy
  A PyTest plugin to speed up your tests which depend on Snowflake sessions
  ☆27Updated 9 months ago
• developer-advocacy-dremio / quick-guides-from-dremio
  Quick Guides from Dremio on Several topics
  ☆64Updated 3 weeks ago
• feluelle / airflint
  Enforce Best Practices for all your Airflow DAGs. ⭐
  ☆92Updated this week
• danielbeach / lakescum
  A Python package to help Databricks Unity Catalog users to read and query Delta Lake tables with Polars, DuckDb, or PyArrow.
  ☆22Updated 7 months ago
• dbt-labs / spark-utils
  Utility functions for dbt projects running on Spark
  ☆31Updated last year