gage-russell / pandas-lineage
☆13Updated 2 years ago
Alternatives and similar repositories for pandas-lineage:
Users that are interested in pandas-lineage are comparing it to the libraries listed below
- Dask integration for Snowflake☆30Updated 3 months ago
- A Giter8 template for scio☆30Updated 2 weeks ago
- Functional Airflow DAG definitions.☆38Updated 7 years ago
- type-class based data cleansing library for Apache Spark SQL☆79Updated 5 years ago
- Lighthouse is a library for data lakes built on top of Apache Spark. It provides high-level APIs in Scala to streamline data pipelines an…☆61Updated 5 months ago
- A Spark-based data comparison tool at scale which facilitates software development engineers to compare a plethora of pair combinations o…☆50Updated last year
- Test suite to document the behavior of Spark☆21Updated 3 years ago
- Airflow plugin to transfer arbitrary files between operators☆78Updated 6 years ago
- Composable filesystem hooks and operators for Apache Airflow.☆17Updated 3 years ago
- Spark functions to run popular phonetic and string matching algorithms☆60Updated 2 years ago
- JSON schema parser for Apache Spark☆81Updated 2 years ago
- Read Delta tables without any Spark☆47Updated 11 months ago
- Scala SDK for working with Snowplow enriched events in Spark, AWS Lambda, Flink et al.☆20Updated 3 months ago
- A Spark datasource for the HadoopOffice library☆38Updated 2 years ago
- CLI tool to launch Spark jobs on AWS EMR☆67Updated last year
- Schema registry for CSV, TSV, JSON, AVRO and Parquet schema. Supports schema inference and GraphQL API.☆111Updated 4 years ago
- Utilities for writing tests that use Apache Spark.☆24Updated 6 years ago
- Data validation library for PySpark 3.0.0☆34Updated 2 years ago
- ETLy is an add-on dashboard service on top of Apache Airflow.☆69Updated last year
- Spark package to "plug" holes in data using SQL based rules ⚡️ 🔌☆28Updated 4 years ago
- Shed light on your data layout in order to monitor the health of your Lakehouse tables and identify when data maintenance operations shou…☆10Updated last year
- Apache Spark AWS Lambda Executor (SAMBA)☆44Updated 6 years ago
- DuckDB with Dashboarding tools demo evidence, streamlit and rill☆15Updated last year
- A pyspark lib to validate data quality☆18Updated 2 years ago
- control spark-shell from vim☆10Updated 8 years ago
- DBT Cloud Plugin for Airflow☆38Updated 9 months ago
- Quickly get a kubernetes executor airflow environment provisioned on GKE. Azure Kubernetes Service instructions included also as are inst…☆36Updated 4 years ago
- Utilities to work with Scala/Java code with py4j☆40Updated last year
- AWS bootstrap scripts for Mozilla's flavoured Spark setup.☆47Updated 5 years ago
- The sane way of building a data layer in Airflow☆24Updated 5 years ago