viaduct-ai / docker-spark-k8s-aws
Docker image for running Spark 3 on Kubernetes on AWS
☆27Updated 3 years ago
Alternatives and similar repositories for docker-spark-k8s-aws:
Users that are interested in docker-spark-k8s-aws are comparing it to the libraries listed below
- The AWS Glue Data Catalog is a fully managed, Apache Hive Metastore compatible, metadata repository. Customers can use the Data Catalog a…☆220Updated last month
- Examples and custom spark images for working with the spark-on-k8s operator on AWS☆26Updated 4 years ago
- Airflow support for Marquez☆32Updated 4 years ago
- ☆80Updated 2 weeks ago
- Helm charts for Trino and Trino Gateway☆165Updated last week
- Spark on Kubernetes infrastructure Helm charts repo☆201Updated 2 years ago
- Performant Redshift data source for Apache Spark☆139Updated 2 weeks ago
- Example for article Running Spark 3 with standalone Hive Metastore 3.0☆98Updated 2 years ago
- Performance optimization for Spark running on Kubernetes☆87Updated 4 years ago
- ☆12Updated 2 months ago
- Setup for running Trino with Hive Metastore on Kubernetes☆101Updated 2 years ago
- Continuously synchronize directories from remote object store to local filesystem☆103Updated 2 months ago
- ☆265Updated 6 months ago
- This code demonstrates the architecture featured on the AWS Big Data blog (https://aws.amazon.com/blogs/big-data/ ) which creates a concu…☆75Updated 6 years ago
- Trino dbt demo project to mix and load BigQuery data with and in a local PostgreSQL database☆75Updated 3 years ago
- Airflow Providers containing Deferrable Operators & Sensors from Astronomer☆148Updated this week
- The Trino (https://trino.io/) adapter plugin for dbt (https://getdbt.com)☆233Updated last month
- Apache Hive Metastore as a Standalone server in Docker☆74Updated 8 months ago
- Sample code to collect Apache Iceberg metrics for table monitoring☆26Updated 8 months ago
- The Amazon S3 Tables catalog is a client library that bridges control plane operations provided by S3 Tables to engines like Apache Spark…☆115Updated 2 months ago
- Adapter for dbt that executes dbt pipelines on Apache Flink☆95Updated last year
- Repository of helm charts for deploying DataHub on a Kubernetes cluster☆183Updated 3 weeks ago
- Spark ETL example processing New York taxi rides public dataset on EKS☆44Updated 2 years ago
- Bare minimal Airflow on Kubernetes (Local, EKS, AKS)☆53Updated 5 years ago
- Performance Observability for Apache Spark☆248Updated last month
- A client for connecting and running DDLs on hive metastore.☆54Updated last year
- Enforce Best Practices for all your Airflow DAGs. ⭐☆99Updated this week
- Spark runtime on AWS Lambda☆107Updated 7 months ago
- The athena adapter plugin for dbt (https://getdbt.com)☆140Updated 2 years ago
- PySpark data-pipeline testing and CICD☆28Updated 4 years ago