bloomberg / apache-spark-on-k8s
Apache Spark enhanced with native Kubernetes scheduler back-end
☆15Updated last year
Alternatives and similar repositories for apache-spark-on-k8s:
Users that are interested in apache-spark-on-k8s are comparing it to the libraries listed below
- Scala SDK for working with Snowplow enriched events in Spark, AWS Lambda, Flink et al.☆20Updated 4 months ago
- The sane way of building a data layer in Airflow☆24Updated 5 years ago
- Export Airflow metrics (from mysql) in prometheus format☆29Updated last month
- ☆11Updated 5 years ago
- Spark ETL example processing New York taxi rides public dataset on EKS☆44Updated 2 years ago
- An Operator for scheduling and executing NiFi Flows as Jobs on Kubernetes☆53Updated 4 years ago
- ☆28Updated 6 months ago
- PySpark for ETL jobs including lineage to Apache Atlas in one script via code inspection☆18Updated 8 years ago
- Dremio Flight connector. Access Dremio using Arrow flight☆40Updated 4 years ago
- Ansible playbooks for Apache Spark on kube☆27Updated 7 years ago
- Deploy Presto on the cloud easily, using Terraform and Packer☆44Updated 2 years ago
- This repository contains recipes for Apache Pinot.☆30Updated last month
- Aiven's S3 Sink Connector for Apache Kafka®☆69Updated 7 months ago
- Fybrik platform - Arrow/Flight module☆16Updated 7 months ago
- Automatically loads new partitions in AWS Athena☆18Updated 4 years ago
- Spark Scala docker container sample for AWS testing - EKS & S3☆24Updated 6 years ago
- Utility functions for dbt projects running on Spark☆31Updated last month
- Skeleton project for Apache Airflow training participants to work on.☆16Updated 4 years ago
- Mirrors a Kinesis stream to Amazon S3 using the KCL☆42Updated 6 months ago
- ⚠️ MAINTENANCE-ONLY MODE: Snowplow maintained SQL data models for working with Snowplow web and mobile behavioral data.☆41Updated 2 months ago
- Apiary provides modules which can be combined to create a federated cloud data lake☆36Updated 11 months ago
- Demos for Nessie. Nessie provides Git-like capabilities for your Data Lake.☆28Updated last week
- Data Profiler for AWS Glue Data Catalog application as described in the AWS Big Data Blog post "Build an automatic data profiling and rep…☆19Updated 4 years ago
- Rokku project. This project acts as a proxy on top of any S3 storage solution providing services like authentication, authorization, shor…☆66Updated last month
- Terraform / NiFi on the Google Cloud Platform☆28Updated 4 months ago
- Testing Scala code with scalatest☆12Updated 2 years ago
- DataHub on AWS demonstration resources☆10Updated 2 years ago
- Ansible roles to deploy Kubernetes, JupyterHub, Jupyter Enterprise Gateway and Spark on Kubernetes cluster☆39Updated 4 years ago
- Apache-Spark based Data Flow(ETL) Framework which supports multiple read, write destinations of different types and also support multiple…☆26Updated 3 years ago
- A facebook for data☆26Updated 5 years ago