GoogleCloudDataproc / spark-bigquery-connectorLinks
BigQuery data source for Apache Spark: Read data from BigQuery into DataFrames, write DataFrames into BigQuery tables.
☆400Updated last week
Alternatives and similar repositories for spark-bigquery-connector
Users that are interested in spark-bigquery-connector are comparing it to the libraries listed below
Sorting:
- dbt-spark contains all of the code enabling dbt to work with Apache Spark and Databricks☆431Updated 3 months ago
- Dataproc templates and pipelines for solving in-cloud data tasks☆129Updated this week
- Data Quality Engine for BigQuery☆273Updated 2 weeks ago
- Reference implementation for real-time Data Lineage tracking for BigQuery using Audit Logs, ZetaSQL and Dataflow.☆143Updated last year
- Snowflake Data Source for Apache Spark.☆226Updated this week
- Libraries and tools for interoperability between Hadoop-related open-source software and Google Cloud Platform.☆285Updated this week
- ☆199Updated last year
- Apache Airflow integration for dbt☆404Updated last year
- A Python Library to support running data quality rules while the spark job is running⚡☆188Updated this week
- Spark style guide☆259Updated 8 months ago
- The Trino (https://trino.io/) adapter plugin for dbt (https://getdbt.com)☆237Updated this week
- Cloud Dataproc: Samples and Utils☆203Updated this week
- CLI that makes it easy to create, test and deploy Airflow DAGs to Astronomer☆387Updated last week
- A library that provides useful extensions to Apache Spark and PySpark.☆224Updated 2 months ago
- This is the development repository for sparkMeasure, a tool and library designed for efficient analysis and troubleshooting of Apache Spa…☆758Updated 3 weeks ago
- Essential Spark extensions and helper methods ✨😲☆760Updated 7 months ago
- Great Expectations Airflow operator☆164Updated last week
- Apache Spark testing helpers (dependency free & works with Scalatest, uTest, and MUnit)☆447Updated this week
- Astronomer Core Docker Images☆107Updated last year
- Airflow Providers containing Deferrable Operators & Sensors from Astronomer☆148Updated last week
- PySpark test helper methods with beautiful error messages☆696Updated last month
- Airflow Backfill UI based plugin for existing / new Airflow environment☆65Updated 4 years ago
- A repository of sample code to accompany our blog post on Airflow and dbt.☆173Updated last year
- dbt-bigquery contains all of the code required to make dbt operate on a BigQuery database.☆251Updated 3 months ago
- Qubole Sparklens tool for performance tuning Apache Spark☆577Updated 11 months ago
- dbt macros to stage external sources☆339Updated last week
- ☆87Updated this week
- A simplified, lightweight ETL Framework based on Apache Spark☆586Updated last year
- Spark data source for Salesforce☆80Updated last year
- Repository of helm charts for deploying DataHub on a Kubernetes cluster☆188Updated this week