Alternatives and similar repositories for spark-pit:

Users that are interested in spark-pit are comparing it to the libraries listed below

• rymurr / flight-spark-source
  ☆105Updated last year
• cerndb / SparkPlugins
  Code and examples of how to write and deploy Apache Spark Plugins. Spark plugins allow runnig custom code on the executors as they are in…
  ☆86Updated 10 months ago
• yaooqinn / itachi
  A library that brings useful functions from various modern database management systems to Apache Spark
  ☆58Updated last year
• SaurabhChawla100 / spark-radiant
  Spark-Radiant is Apache Spark Performance and Cost Optimizer
  ☆25Updated last month
• oap-project / remote-shuffle
  Spark* shuffle plugin for support shuffling data through a remote Hadoop-compatible file system, as opposed to vanilla Spark's local-dis…
  ☆21Updated 11 months ago
• apache / datafusion-benchmarks
  Apache DataFusion Benchmarks
  ☆16Updated 3 months ago
• oap-project / sql-ds-cache
  Spark* plug-in for accelerating Spark* SQL performance by using cache and index at SQL data source layer.
  ☆37Updated 2 years ago
• G-Research / spark-extension
  A library that provides useful extensions to Apache Spark and PySpark.
  ☆214Updated 2 months ago
• apache / datasketches
  Apache datasketches
  ☆94Updated 2 years ago
• findify / featury
  Friendly ML feature store
  ☆45Updated 2 years ago
• holdenk / spark-upgrade
  Magic to help Spark pipelines upgrade
  ☆34Updated 4 months ago
• substrait-io / substrait-java
  ☆80Updated last week
• voltrondata / spark-substrait-gateway
  Implements a gateway that speaks the SparkConnect protocol and drives a backend using Substrait (over ADBC Flight SQL).
  ☆16Updated last week
• delta-incubator / deltaray
  Delta reader for the Ray open-source toolkit for building ML applications
  ☆45Updated last year
• qwshen / spark-flight-connector
  A Spark Connector that reads data from / writes data to Arrow-Flight end-points with Arrow-Flight and Flight-SQL
  ☆39Updated 4 months ago
• BryanCutler / SparkArrowFlight
  Example for simple Apache Arrow Flight service with Apache Spark and TensorFlow clients
  ☆36Updated 3 years ago
• airbnb / sputnik
  ☆63Updated 5 years ago
• squito / spark-memory
  A tool to get better debug info on spark's memory usage
  ☆42Updated 5 years ago
• ray-project / deltacat
  A portable Pythonic Data Lakehouse powered by Ray that brings exabyte-level scalability and fast, ACID-compliant, change-data-capture to …
  ☆190Updated this week
• lightcopy / parquet-index
  Spark SQL index for Parquet tables
  ☆134Updated 3 years ago
• paypal / dione
  Dione - a Spark and HDFS indexing library
  ☆51Updated 11 months ago
• criteo / cluster-pack
  A library on top of either pex or conda-pack to make your Python code easily available on a cluster
  ☆45Updated 2 months ago
• projectnessie / iceberg-catalog-migrator
  CLI tool to bulk migrate the tables from one catalog another without a data copy
  ☆75Updated this week
• lhbench / lhbench
  Lakehouse storage system benchmark
  ☆70Updated last year
• maropu / datasketches-spark
  Data Sketches for Apache Spark
  ☆22Updated 2 years ago
• xskipper-io / xskipper
  An Extensible Data Skipping Framework
  ☆43Updated last month
• maropu / spark-tpcds-datagen
  All the things about TPC-DS in Apache Spark
  ☆104Updated last year
• TU-Berlin-DIMA / scotty-window-processor
  This repository provides Scotty, a framework for efficient window aggregations for out-of-order Stream Processing.
  ☆76Updated last year
• microsoft / lst-bench
  LST-Bench is a framework that allows users to run benchmarks specifically designed for evaluating Log-Structured Tables (LSTs) such as De…
  ☆72Updated 2 weeks ago
• maropu / spark-sql-flow-plugin
  Visualize column-level data lineage in Spark SQL
  ☆88Updated 2 years ago