GoogleCloudDataproc / custom-images
Tools for creating Dataproc custom images
☆32Updated 2 weeks ago
Alternatives and similar repositories for custom-images:
Users that are interested in custom-images are comparing it to the libraries listed below
- ☆46Updated 9 months ago
- ☆54Updated 7 years ago
- Sample code with integration between Data Catalog and Hive data source.☆25Updated 2 weeks ago
- Oozie Workflow to Airflow DAGs migration tool☆88Updated last month
- Stream Avro SpecificRecord objects in BigQuery using Cloud Dataflow☆13Updated 3 years ago
- A Python package to centralize some Google Cloud Data Catalog scripts, this repo contains commands like bulk CSV operations that help lev…☆21Updated 2 years ago
- Cloud Dataproc: Samples and Utils☆200Updated last month
- Example Spark applications that run on Kubernetes and access GCP products, e.g., GCS, BigQuery, and Cloud PubSub☆37Updated 7 years ago
- Sample code with integration between Data Catalog and BI data sources.☆32Updated 3 years ago
- Solution Accelerators for Serverless Spark on GCP, the industry's first auto-scaling and serverless Spark as a service☆64Updated 9 months ago
- Multi Cloud Data Tokenization Solution By Using Dataflow and Cloud DLP☆90Updated 6 months ago
- Data validation library for PySpark 3.0.0☆34Updated 2 years ago
- Automatically discover and tag PII data across BigQuery tables and apply column-level access controls based on confidentiality level.☆49Updated 3 weeks ago
- Rules based grant management for Snowflake☆40Updated 6 years ago
- Magic to help Spark pipelines upgrade☆34Updated 4 months ago
- Repository with examples and smoke tests for the GCP Airflow operators and hooks☆146Updated 8 years ago
- This library has moved to https://github.com/googleapis/google-cloud-python/tree/main/packages/google-cloud-dataproc☆48Updated last year
- An example PySpark project with pytest☆17Updated 7 years ago
- ☕⛵WIP PySpark dependency management☆22Updated 6 years ago
- hive_compared_bq compares/validates 2 (SQL like) tables, and graphically shows the rows/columns that are different.☆28Updated 7 years ago
- This is the support code and solutions for the NYC Taxi Tycoon Dataflow Codelab☆60Updated 5 years ago
- Data Catalog Tag Templates☆30Updated 4 months ago
- Example for article Running Spark 3 with standalone Hive Metastore 3.0☆97Updated 2 years ago
- The Internals of Spark on Kubernetes☆70Updated 2 years ago
- Repo with scripts and automation to help ensure best practices in Google Data Catalog☆13Updated 3 years ago
- ☆31Updated 6 years ago
- Airflow on Kubernetes Operator☆89Updated 2 years ago
- Reference implementation for real-time Data Lineage tracking for BigQuery using Audit Logs, ZetaSQL and Dataflow.☆142Updated 8 months ago
- ☆65Updated 5 months ago
- A repository of sample code to show data quality checking best practices using Airflow.☆74Updated last year