Alternatives and similar repositories for spark-submit

Users that are interested in spark-submit are comparing it to the libraries listed below

• trinodb / trino-python-client
  Python client for Trino
  ☆411Updated 5 months ago
• Nike-Inc / spark-expectations
  A Python Library to support running data quality rules while the spark job is running⚡
  ☆197Updated this week
• Nike-Inc / brickflow
  Pythonic Programming Framework to orchestrate jobs in Databricks Workflow
  ☆226Updated this week
• eakmanrq / sqlframe
  Turning PySpark Into a Universal DataFrame API
  ☆485Updated this week
• G-Research / spark-extension
  A library that provides useful extensions to Apache Spark and PySpark.
  ☆232Updated 2 weeks ago
• MrPowers / chispa
  PySpark test helper methods with beautiful error messages
  ☆752Updated 3 weeks ago
• MrPowers / mack
  Delta Lake helper methods in PySpark
  ☆327Updated 3 weeks ago
• delta-io / delta-examples
  Delta Lake examples
  ☆238Updated last year
• great-expectations / airflow-provider-great-expectations
  Great Expectations Airflow operator
  ☆170Updated last week
• sfu-db / lineagex
  ☆81Updated 11 months ago
• starburstdata / dbt-trino
  The Trino (https://trino.io/) adapter plugin for dbt (https://getdbt.com)
  ☆253Updated 3 weeks ago
• mrpowers-io / levi
  Delta Lake helper methods. No Spark dependency.
  ☆22Updated 3 weeks ago
• hussein-awala / spark-on-k8s
  A Python package to submit and manage Apache Spark applications on Kubernetes.
  ☆46Updated 6 months ago
• awslabs / python-deequ
  Python API for Deequ
  ☆809Updated 2 weeks ago
• 1ambda / lakehouse
  Playground for Lakehouse (Iceberg, Hudi, Spark, Flink, Trino, DBT, Airflow, Kafka, Debezium CDC)
  ☆64Updated 2 years ago
• bitsondatadev / trino-getting-started
  ☆269Updated last year
• exacaster / lighter
  REST API for Apache Spark on K8S or YARN
  ☆109Updated 2 months ago
• astronomer / astro-sdk
  Astro SDK allows rapid and clean development of {Extract, Load, Transform} workflows using Python and SQL, powered by Apache Airflow.
  ☆375Updated 8 months ago
• dataflint / spark
  Drop-in replacement for Apache Spark UI
  ☆401Updated this week
• tokern / data-lineage
  Generate and Visualize Data Lineage from query history
  ☆328Updated 2 years ago
• mitchelllisle / sparkdantic
  ✨ A Pydantic to PySpark schema library
  ☆118Updated 2 weeks ago
• getindata / dbt-flink-adapter
  Adapter for dbt that executes dbt pipelines on Apache Flink
  ☆96Updated last year
• andreax79 / airflow-code-editor
  A plugin for Apache Airflow that allows you to edit DAGs in browser
  ☆458Updated 2 weeks ago
• dbt-labs / dbt-spark
  This repository has moved into https://github.com/dbt-labs/dbt-adapters
  ☆443Updated 6 months ago
• apache / iceberg-python
  PyIceberg
  ☆993Updated this week
• ananthdurai / schemata
  Schema modelling framework for decentralised domain-driven ownership of data.
  ☆261Updated 2 years ago
• rafaelpierre / pyjaws
  PyJaws: A Pythonic Way to Define Databricks Jobs and Workflows
  ☆45Updated 2 weeks ago
• dbt-labs / dbt-athena
  This repository has moved into https://github.com/dbt-labs/dbt-adapters
  ☆250Updated last year
• pipeline-tools / gusty
  Making DAG construction easier
  ☆283Updated last month
• astronomer / astronomer-providers
  Airflow Providers containing Deferrable Operators & Sensors from Astronomer
  ☆149Updated 2 weeks ago