A lightweight, declarative PySpark framework for data quality validation — check columns, rows, and entire datasets directly in your Spark pipelines
☆78Sep 3, 2026Updated 2 weeks ago
Alternatives and similar repositories for sparkdq
Users that are interested in sparkdq are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Full-stack OpenTelemetry observability for Apache Spark☆18Sep 12, 2026Updated last week
- Streaming PDF processor for Scala☆13Apr 2, 2025Updated last year
- An R package for easy cohort analysis with event data☆13Jul 9, 2026Updated 2 months ago
- Python dataflow pipeline☆10May 12, 2019Updated 7 years ago
- ETL jobs for Firefox Telemetry☆29Aug 18, 2026Updated last month
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- Apple app store reviews and ratings scraper. Use LLM, unmaintained.☆12Aug 2, 2025Updated last year
- ☆13May 12, 2026Updated 4 months ago
- Collection of functions for common business intelligence related analysis☆15Aug 8, 2026Updated last month
- ☆49May 14, 2025Updated last year
- This project supplies utility classes and functions for dynamic loading and management of data from designated directories. As well as co…☆16Mar 5, 2024Updated 2 years ago
- example BQ data model using Dataform☆17Updated this week
- A Scala SDK for interfacing with HashiCorp's Nomad☆18Nov 30, 2022Updated 3 years ago
- Kubernetes operator for Apache Superset. Deploy and run Superset with the Stackable Data Platform (SDP).☆35Updated this week
- A leightweight UI for Lakekeeper☆20Updated this week
- Wordpress hosting with auto-scaling - Free Trial Offer • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- Manage and synchronize Synapse notebooks with Jupyter notebooks.☆11Nov 30, 2023Updated 2 years ago
- Provides domain models and codecs for common protocols☆28Oct 23, 2021Updated 4 years ago
- A sample project combining Microsoft Fabric Lakehouse, DuckLake and SQLMesh☆16Jun 24, 2025Updated last year
- A python SPark ETL libRary (SPETLR) for Databricks. https://discord.gg/p9bzqGybVW☆24Jul 24, 2026Updated last month
- From linear regression towards neural networks...☆27Dec 31, 2025Updated 8 months ago
- R package: make pixel art interactively in a plot window, get a matrix, make a gif☆24Jun 4, 2024Updated 2 years ago
- Samples for fabric user data functions☆29Updated this week
- A full end-to-end serverless pipeline to read Tweets and send them to BigQuery and Firestore/Firebase☆14Sep 6, 2020Updated 6 years ago
- Cost Efficient Data Pipelines with DuckDB☆63May 14, 2025Updated last year
- End-to-end encrypted email - Proton Mail • AdSpecial offer: 40% Off Yearly / 80% Off First Month. All Proton services are open source and independently audited for security.
- Spark fires is a anti-pattern playground where we deliberately break Spark applications in various ways so you can observe what happens a…☆42Nov 18, 2024Updated last year
- Examples of building APIs in R using Plumber☆24Nov 12, 2021Updated 4 years ago
- R package to wrap the Deutsche Bahn Fahrplan API☆17Feb 4, 2024Updated 2 years ago
- ☆41Nov 19, 2021Updated 4 years ago
- Python library for calculating psychrometric states of moist air.☆19Mar 19, 2017Updated 9 years ago
- Python API for Deequ☆826Updated this week
- A back-end agnostic spatial data frame inspired by rust trait implementations☆28Updated this week
- Databricks framework to validate Data Quality of pySpark DataFrames and Tables☆459Updated this week
- This is an experimental project that seeks to integrate PySpark and xarray for Climate Data Analysis.☆21Nov 16, 2017Updated 8 years ago
- Wordpress hosting with auto-scaling - Free Trial Offer • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- ☆30Aug 2, 2026Updated last month
- Point-in-Time optimizations for Apache Spark☆30Jan 18, 2024Updated 2 years ago
- Cross-platform .NET MAUI plugin to read & write health and fitness data via Apple HealthKit (iOS) and Android Health Connect.☆14Jul 1, 2026Updated 2 months ago
- Stream content to/from an SFTP Server☆15Aug 16, 2022Updated 4 years ago
- Code for "Efficient Data Processing in Spark" Course☆393Jun 29, 2026Updated 2 months ago
- protobuf pyspark conversion☆23Jun 7, 2023Updated 3 years ago
- Code to demonstrate data engineering metadata & logging best practices☆21Mar 12, 2024Updated 2 years ago