bloomberg / apache-spark-on-k8sLinks
Apache Spark enhanced with native Kubernetes scheduler back-end
☆15Updated 2 years ago
Alternatives and similar repositories for apache-spark-on-k8s
Users that are interested in apache-spark-on-k8s are comparing it to the libraries listed below
Sorting:
- Ansible roles to deploy Kubernetes, JupyterHub, Jupyter Enterprise Gateway and Spark on Kubernetes cluster☆37Updated 5 years ago
- Export Airflow metrics (from mysql) in prometheus format☆29Updated 8 months ago
- ☆36Updated 9 months ago
- ☆10Updated 3 years ago
- Scala SDK for working with Snowplow enriched events in Spark, AWS Lambda, Flink et al.☆21Updated last year
- Apache Airflow CI pipeline☆19Updated 6 years ago
- The sane way of building a data layer in Airflow☆24Updated 6 years ago
- Skeleton project for Apache Airflow training participants to work on.☆17Updated 5 years ago
- Ansible playbooks for Apache Spark on kube☆27Updated 8 years ago
- Fybrik platform - Arrow/Flight module☆16Updated last year
- Airflow workflow management platform chef cookbook.☆70Updated 6 years ago
- A Giter8 template for scio☆31Updated last month
- Example Spark applications that run on Kubernetes and access GCP products, e.g., GCS, BigQuery, and Cloud PubSub☆37Updated 7 years ago
- A python wrapper for the KSQL REST API.☆158Updated 2 years ago
- A facebook for data☆26Updated 6 years ago
- Superglue is a lineage-tracking tool built to help visualize the propagation of data through complex pipelines composed of tables, jobs …☆159Updated 3 years ago
- Oozie Workflow to Airflow DAGs migration tool☆89Updated last month
- Event data simulator. Generates a stream of pseudo-random events from a set of users, designed to simulate web traffic.☆28Updated 8 years ago
- A Spark-based data comparison tool at scale which facilitates software development engineers to compare a plethora of pair combinations o…☆52Updated 6 months ago
- An example PySpark project with pytest☆17Updated 8 years ago
- Dremio Flight connector. Access Dremio using Arrow flight☆39Updated 5 years ago
- dbt adapter for connecting to MindsDB☆19Updated last year
- Debussy is an opinionated Data Architecture and Engineering framework, enabling data analysts and engineers to build better platforms and…☆28Updated 2 years ago
- Demos for Nessie. Nessie provides Git-like capabilities for your Data Lake.☆30Updated 2 weeks ago
- Herd-UI is a search and discovery tool for business and technical users. Everyone in your organization can use Herd-UI to browse and unde…☆16Updated 3 years ago
- Airflow declarative DAGs via YAML☆133Updated 2 years ago
- Spark Scala docker container sample for AWS testing - EKS & S3☆24Updated 7 years ago
- type-class based data cleansing library for Apache Spark SQL☆78Updated 6 years ago
- Data validation library for PySpark 3.0.0☆33Updated 3 years ago
- A Helm Chart for Apache Airflow☆14Updated 7 years ago