Alternatives and similar repositories for k8s-big-data-experiments

Users that are interested in k8s-big-data-experiments are comparing it to the libraries listed below

• rssanders3 / airflow-spark-operator-plugin
  A plugin to Apache Airflow to allow you to run Spark Submit Commands as an Operator
  ☆73Updated 6 years ago
• rambler-digital-solutions / airflow-declarative
  Airflow declarative DAGs via YAML
  ☆133Updated 2 years ago
• godatadriven / whirl
  Fast iterative local development and testing of Apache Airflow workflows
  ☆202Updated 3 months ago
• mozilla / emr-bootstrap-spark
  AWS bootstrap scripts for Mozilla's flavoured Spark setup.
  ☆47Updated 5 years ago
• GoogleCloudPlatform / oozie-to-airflow
  Oozie Workflow to Airflow DAGs migration tool
  ☆88Updated 8 months ago
• Wikia / discreETLy
  ETLy is an add-on dashboard service on top of Apache Airflow.
  ☆68Updated 2 years ago
• snowplow / snowplow-scala-analytics-sdk
  Scala SDK for working with Snowplow enriched events in Spark, AWS Lambda, Flink et al.
  ☆21Updated last year
• funkyminds / cleanframes
  type-class based data cleansing library for Apache Spark SQL
  ☆78Updated 6 years ago
• acroz / pylivy
  A Python client for Apache Livy, enabling use of remote Apache Spark clusters.
  ☆70Updated 3 years ago
• mikulskibartosz / check-engine
  Data validation library for PySpark 3.0.0
  ☆33Updated 3 years ago
• yaooqinn / itachi
  A library that brings useful functions from various modern database management systems to Apache Spark
  ☆60Updated 2 years ago
• ing-bank / rokku
  Rokku project. This project acts as a proxy on top of any S3 storage solution providing services like authentication, authorization, shor…
  ☆70Updated 2 months ago
• airflow-plugins / airflow_api_plugin
  REST-like API exposing Airflow data and operations
  ☆61Updated 6 years ago
• bahchis / airflow-cookbook
  Airflow workflow management platform chef cookbook.
  ☆71Updated 6 years ago
• MarquezProject / marquez-python
  Python client for Marquez
  ☆12Updated 4 years ago
• Lewuathe / docker-trino-cluster
  Multiple node presto cluster on docker container
  ☆126Updated 3 years ago
• palantir / spark-influx-sink
  A Spark metrics sink that pushes to InfluxDb
  ☆51Updated 4 years ago
• FINRAOS / MegaSparkDiff
  A Spark-based data comparison tool at scale which facilitates software development engineers to compare a plethora of pair combinations o…
  ☆52Updated 5 months ago
• GoogleCloudDataproc / spark-spanner-connector
  Cloud Spanner Connector for Apache Spark
  ☆17Updated 10 months ago
• BrechtDeVlieger / airflow-kube-helm
  ☆128Updated 5 years ago
• hortonworks-spark / spark-schema-registry
  Schema Registry integration for Apache Spark
  ☆40Updated 3 years ago
• zheyuan28 / SparkTaskMetrics
  Task Metrics Explorer
  ☆14Updated 6 years ago
• zero323 / pyspark-stubs
  Apache (Py)Spark type annotations (stub files).
  ☆118Updated 3 years ago
• FRosner / drunken-data-quality
  Spark package for checking data quality
  ☆222Updated 5 years ago
• etsy / boundary-layer
  Builds Airflow DAGs from configuration files. Powers all DAGs on the Etsy Data Platform
  ☆260Updated 2 years ago
• dsaidgovsg / airflow-pipeline
  An Airflow docker image preconfigured to work well with Spark and Hadoop/EMR
  ☆175Updated 5 months ago
• jrderuiter / airflow-fs
  Composable filesystem hooks and operators for Apache Airflow.
  ☆17Updated 4 years ago
• blue-yonder / airflow-plugin-demo
  Example for an airflow plugin
  ☆49Updated 9 years ago
• target / data-validator
  A tool to validate data, built around Apache Spark.
  ☆100Updated this week
• aseigneurin / spark-ui-proxy
  Lightweight proxy to expose the UI of an Apache Spark cluster that is behind a firewall
  ☆98Updated 5 years ago