GoogleCloudDataproc / spark-bigquery-connectorLinks
BigQuery data source for Apache Spark: Read data from BigQuery into DataFrames, write DataFrames into BigQuery tables.
☆401Updated this week
Alternatives and similar repositories for spark-bigquery-connector
Users that are interested in spark-bigquery-connector are comparing it to the libraries listed below
Sorting:
- Dataproc templates and pipelines for solving in-cloud data tasks☆129Updated this week
- dbt-spark contains all of the code enabling dbt to work with Apache Spark and Databricks☆433Updated 4 months ago
- Data Quality Engine for BigQuery☆275Updated last month
- Snowflake Data Source for Apache Spark.☆226Updated last week
- Libraries and tools for interoperability between Hadoop-related open-source software and Google Cloud Platform.☆285Updated this week
- Reference implementation for real-time Data Lineage tracking for BigQuery using Audit Logs, ZetaSQL and Dataflow.☆144Updated last year
- Cloud Dataproc: Samples and Utils☆203Updated last week
- Spark style guide☆259Updated 8 months ago
- Apache Spark testing helpers (dependency free & works with Scalatest, uTest, and MUnit)☆447Updated 2 weeks ago
- Airflow Unit Tests and Integration Tests☆259Updated 2 years ago
- ☆199Updated last year
- Astronomer Core Docker Images☆107Updated last year
- A Python Library to support running data quality rules while the spark job is running⚡☆188Updated last week
- Apache Airflow integration for dbt☆405Updated last year
- Avro SerDe for Apache Spark structured APIs.☆236Updated 2 weeks ago
- PySpark test helper methods with beautiful error messages☆699Updated 2 weeks ago
- A repository of sample code to accompany our blog post on Airflow and dbt.☆174Updated last year
- CLI that makes it easy to create, test and deploy Airflow DAGs to Astronomer☆393Updated this week
- Solution Accelerators for Serverless Spark on GCP, the industry's first auto-scaling and serverless Spark as a service☆69Updated last year
- Great Expectations Airflow operator☆166Updated this week
- Airflow Backfill UI based plugin for existing / new Airflow environment☆65Updated 4 years ago
- A library that provides useful extensions to Apache Spark and PySpark.☆226Updated 3 months ago
- Essential Spark extensions and helper methods ✨😲☆761Updated 8 months ago
- dbt-bigquery contains all of the code required to make dbt operate on a BigQuery database.☆252Updated 4 months ago
- Airflow Providers containing Deferrable Operators & Sensors from Astronomer☆149Updated last week
- pyspark methods to enhance developer productivity 📣 👯 🎉☆672Updated 3 months ago
- A simplified, lightweight ETL Framework based on Apache Spark☆586Updated last year
- Data pipeline with dbt, Airflow, Great Expectations☆163Updated 3 years ago
- Data ingestion library for Amundsen to build graph and search index☆205Updated last year
- An end to end demo of Google's Cloud data and analytic stack.☆253Updated last week