Alternatives and similar repositories for spark

Users that are interested in spark are comparing it to the libraries listed below

• apache-spark-on-k8s / kubernetes-HDFS

  View on GitHub
  Repository holding configuration files for running an HDFS cluster in Kubernetes
  ☆398Sep 25, 2024Updated last year
• apache-spark-on-k8s / ansible

  View on GitHub
  Ansible playbooks for Apache Spark on kube
  ☆27Jul 20, 2017Updated 8 years ago
• kubeflow / spark-operator

  View on GitHub
  Kubernetes operator for managing the lifecycle of Apache Spark applications on Kubernetes.
  ☆3,100Feb 6, 2026Updated last week
• kubernetes-retired / application-images

  View on GitHub
  [EOL] Image build contents for Kubernetes applications.
  ☆47Jan 22, 2018Updated 8 years ago
• Comcast / kube-yarn

  View on GitHub
  Running YARN on Kubernetes with PetSet controller.
  ☆166Feb 22, 2018Updated 7 years ago
• cloudera / livy

  View on GitHub
  Livy is an open source REST interface for interacting with Apache Spark from anywhere
  ☆1,007Oct 5, 2022Updated 3 years ago
• palantir / k8s-spark-scheduler

  View on GitHub
  A Kubernetes Scheduler Extender to provide gang scheduling support for Spark on Kubernetes
  ☆177Apr 23, 2023Updated 2 years ago
• krallistic / kafka-operator

  View on GitHub
  A Kafka Operator for Kubernetes
  ☆294Oct 28, 2018Updated 7 years ago
• spark-jobserver / spark-jobserver

  View on GitHub
  REST job server for Apache Spark
  ☆2,845Jul 8, 2025Updated 7 months ago
• kubernetes-retired / kube-batch

  View on GitHub
  A batch scheduler of kubernetes for high performance workload, e.g. AI/ML, BigData, HPC
  ☆1,094May 22, 2023Updated 2 years ago
• yahoo / TensorFlowOnSpark

  View on GitHub
  TensorFlowOnSpark brings TensorFlow programs to Apache Spark clusters.
  ☆3,859Jul 10, 2023Updated 2 years ago
• jupyterhub / kubespawner

  View on GitHub
  Kubernetes spawner for JupyterHub
  ☆594Feb 2, 2026Updated last week
• TIBCOSoftware / snappydata

  View on GitHub
  Project SnappyData - memory optimized analytics database, based on Apache Spark™ and Apache Geode™. Stream, Transact, Analyze, Predict in…
  ☆1,035Nov 21, 2022Updated 3 years ago
• radanalyticsio / spark-operator

  View on GitHub
  Operator for managing the Spark clusters on Kubernetes and OpenShift.
  ☆159Nov 18, 2021Updated 4 years ago
• radanalyticsio / oshinko-webui

  View on GitHub
  Web console for a spark cluster management app
  ☆28Dec 31, 2025Updated last month
• kubeflow / trainer

  View on GitHub
  Distributed AI Model Training and LLM Fine-Tuning on Kubernetes
  ☆2,028Updated this week
• radanalyticsio / openshift-spark

  View on GitHub
  ☆72Nov 17, 2021Updated 4 years ago
• lightbend / flink-k8s-operator

  View on GitHub
  An example of building kubernetes operator (Flink) using Abstract operator's framework
  ☆26Jul 12, 2019Updated 6 years ago
• bloomberg / apache-spark-on-k8s

  View on GitHub
  Apache Spark enhanced with native Kubernetes scheduler back-end
  ☆15Aug 21, 2023Updated 2 years ago
• kubeflow / kubeflow

  View on GitHub
  Machine Learning Toolkit for Kubernetes
  ☆15,446Jan 5, 2026Updated last month
• kubernetes-retired / poseidon

  View on GitHub
  [EOL] A Firmament-based Kubernetes scheduler
  ☆409Jul 19, 2021Updated 4 years ago
• TIBCOSoftware / snappy-on-k8s

  View on GitHub
  An Integrated and collaborative cloud environment for building and running Spark applications on PKS/Kubernetes
  ☆84Mar 16, 2020Updated 5 years ago
• kubernetes-retired / contrib

  View on GitHub
  [EOL] This is a place for various components in the Kubernetes ecosystem that aren't part of the Kubernetes core.
  ☆2,446Apr 17, 2019Updated 6 years ago
• holdenk / spark-testing-base

  View on GitHub
  Base classes to use when writing tests with Spark
  ☆1,549Dec 22, 2025Updated last month
• Hydrospheredata / mist

  View on GitHub
  Serverless proxy for Spark cluster
  ☆324Oct 29, 2020Updated 5 years ago
• kubernetes-retired / heapster

  View on GitHub
  [EOL] Compute Resource Usage Analysis and Monitoring of Container Clusters
  ☆2,720Nov 30, 2018Updated 7 years ago
• apache / incubator-toree

  View on GitHub
  Mirror of Apache Toree (Incubating)
  ☆749Feb 7, 2026Updated last week
• Alluxio / alluxio

  View on GitHub
  Alluxio, data orchestration for analytics and machine learning in the cloud
  ☆7,157Apr 29, 2025Updated 9 months ago
• aseigneurin / spark-ui-proxy

  View on GitHub
  Lightweight proxy to expose the UI of an Apache Spark cluster that is behind a firewall
  ☆98Jul 12, 2020Updated 5 years ago
• amplab / drizzle-spark

  View on GitHub
  Drizzle integration with Apache Spark
  ☆120Sep 11, 2018Updated 7 years ago
• rook / operator-kit

  View on GitHub
  A library for creating a Kubernetes Operator
  ☆278Oct 5, 2021Updated 4 years ago
• kubernetes-retired / service-catalog

  View on GitHub
  Consume services in Kubernetes using the Open Service Broker API
  ☆1,042Feb 1, 2022Updated 4 years ago
• volcano-sh / volcano

  View on GitHub
  A Cloud Native Batch System (Project under CNCF)
  ☆5,320Updated this week
• virtual-kubelet / virtual-kubelet

  View on GitHub
  Virtual Kubelet is an open source Kubernetes kubelet implementation.
  ☆4,480Updated this week
• h2oai / sparkling-water

  View on GitHub
  Sparkling Water provides H2O functionality inside Spark cluster
  ☆977Nov 5, 2025Updated 3 months ago
• apache / incubator-livy

  View on GitHub
  Apache Livy is an open source REST interface for interacting with Apache Spark from anywhere.
  ☆944Updated this week
• alibaba / SparkCube

  View on GitHub
  SparkCube is an open-source project for extremely fast OLAP data analysis. SparkCube is an extension of Apache Spark.
  ☆136Mar 6, 2023Updated 2 years ago
• apache / bahir

  View on GitHub
  Mirror of Apache Bahir
  ☆335Jul 7, 2023Updated 2 years ago
• apache / spark

  View on GitHub
  Apache Spark - A unified analytics engine for large-scale data processing
  ☆42,810Updated this week