Alternatives and similar repositories for spark-distcp

Users that are interested in spark-distcp are comparing it to the libraries listed below

• ExpediaGroup / shunting-yard

  View on GitHub
  Shunting Yard is a real-time data replication tool that copies data between Hive Metastores.
  ☆20Oct 11, 2021Updated 4 years ago
• apache / uniffle

  View on GitHub
  Uniffle is a high performance, general purpose Remote Shuffle Service.
  ☆443Jan 30, 2026Updated 2 weeks ago
• oracle / spark-oracle

  View on GitHub
  On the fly, translation of Spark programs to run natively on your Oracle DB. Your Spark programs require no changes.
  ☆35Apr 15, 2025Updated 9 months ago
• criteo / garmadon

  View on GitHub
  Java event logs collector for hadoop and frameworks
  ☆41Mar 25, 2025Updated 10 months ago
• oap-project / sql-ds-cache

  View on GitHub
  Spark* plug-in for accelerating Spark* SQL performance by using cache and index at SQL data source layer.
  ☆37Jan 3, 2023Updated 3 years ago
• IBM / spark-s3-shuffle

  View on GitHub
  A S3 Shuffle plugin for Apache Spark to enable elastic scaling for generic Spark workloads.
  ☆52Sep 17, 2025Updated 4 months ago
• apache / kyuubi-client

  View on GitHub
  Client libraries of end users of Apache Kyuubi
  ☆11Jan 10, 2023Updated 3 years ago
• ExpediaGroup / waggle-dance

  View on GitHub
  Hive federation service. Enables disparate tables to be concurrently accessed across multiple Hive deployments.
  ☆285Nov 26, 2025Updated 2 months ago
• boostscale / velox4j

  View on GitHub
  Community Java bindings for https://github.com/facebookincubator/velox
  ☆39Feb 8, 2026Updated last week
• criteo / babar

  View on GitHub
  Profiler for large-scale distributed java applications (Spark, Scalding, MapReduce, Hive,...) on YARN.
  ☆128Sep 7, 2018Updated 7 years ago
• marcelmay / hfsa

  View on GitHub
  Hadoop FSImage Analyzer (HFSA)
  ☆66Updated this week
• ClickHouse / spark-clickhouse-connector

  View on GitHub
  Spark ClickHouse Connector build on DataSourceV2 API
  ☆211Feb 1, 2026Updated last week
• Tencent / Firestorm

  View on GitHub
  Firestorm is a Remote Shuffle Service, and provides the capability for Apache Spark and Apache Hadoop MapReduce applications to store shu…
  ☆257Apr 7, 2023Updated 2 years ago
• hxjcarrie / pyspark_study

  View on GitHub
  ☆15Jan 19, 2020Updated 6 years ago
• frb502 / spark-skewed-join-hint

  View on GitHub
  SparkSQL自定义Hint优化器解决热点数据导致JOIN数据倾斜问题
  ☆48Jan 4, 2019Updated 7 years ago
• timveil / hive-jdbc-driver

  View on GitHub
  An alternative to the "hive standalone" jar for connecting Java applications to Apache Hive via JDBC
  ☆41Oct 1, 2024Updated last year
• apache / celeborn

  View on GitHub
  Apache Celeborn is an elastic and high-performance service for shuffle and spilled data.
  ☆1,037Feb 5, 2026Updated last week
• zuston / riffle

  View on GitHub
  Rust based high-performance Apache Uniffle shuffle-server
  ☆60Feb 4, 2026Updated last week
• wankunde / sql-runner

  View on GitHub
  ☆17Mar 19, 2024Updated last year
• liquidm / druid-dumbo

  View on GitHub
  ☆21Mar 17, 2023Updated 2 years ago
• xskipper-io / xskipper

  View on GitHub
  An Extensible Data Skipping Framework
  ☆48Jul 15, 2025Updated 6 months ago
• bitlap / bitlap

  View on GitHub
  A table schema-less OLAP Analytics Engine for Big Data.
  ☆24Apr 23, 2024Updated last year
• amundsen-io / amundsengremlin

  View on GitHub
  Amundsen Gremlin
  ☆22Aug 26, 2022Updated 3 years ago
• shanyu / hadooplogparser

  View on GitHub
  Hadoop Yarn aggregated log parser utility
  ☆23Feb 1, 2020Updated 6 years ago
• dataArtisans / sql-training

  View on GitHub
  ☆39May 13, 2019Updated 6 years ago
• LucaCanali / sparkMeasure

  View on GitHub
  This is the development repository for sparkMeasure, a tool and library designed for efficient analysis and troubleshooting of Apache Spa…
  ☆809Feb 5, 2026Updated last week
• oap-project / gazelle_plugin

  View on GitHub
  Native SQL Engine plugin for Spark SQL with vectorized SIMD optimizations.
  ☆258Feb 21, 2023Updated 2 years ago
• trinodb / tpcds

  View on GitHub
  Port of TPC-DS dsdgen to Java
  ☆22Nov 29, 2022Updated 3 years ago
• maropu / spark-sql-flow-plugin

  View on GitHub
  Visualize column-level data lineage in Spark SQL
  ☆92May 13, 2022Updated 3 years ago
• ververica / ForSt

  View on GitHub
  A Persistent Key-Value Store designed for Streaming processing
  ☆119Jan 13, 2026Updated last month
• linkedin / transport

  View on GitHub
  A framework for writing performant user-defined functions (UDFs) that are portable across a variety of engines including Apache Spark, Ap…
  ☆303Oct 30, 2025Updated 3 months ago
• apache / auron

  View on GitHub
  The Auron accelerator for distributed computing framework (e.g., Spark) leverages native vectorized execution to accelerate query process…
  ☆1,708Updated this week
• qubole / sparklens

  View on GitHub
  Qubole Sparklens tool for performance tuning Apache Spark
  ☆589Jun 26, 2024Updated last year
• yaooqinn / itachi

  View on GitHub
  A library that brings useful functions from various modern database management systems to Apache Spark
  ☆61Sep 4, 2023Updated 2 years ago
• apache / amoro

  View on GitHub
  Apache Amoro(incubating) is a Lakehouse management system built on open data lake formats.
  ☆1,109Updated this week
• Intel-bigdata / SSM

  View on GitHub
  Smart Storage Management for Big Data, a comprehensive hot/cold data optimized solution
  ☆140Jan 3, 2023Updated 3 years ago
• NetEase / spark-alarm

  View on GitHub
  Alerting and monitoring tool for Apache Spark
  ☆23May 20, 2022Updated 3 years ago
• autodeployai / pypmml-spark

  View on GitHub
  Python PMML scoring library for PySpark as SparkML Transformer
  ☆22Dec 9, 2024Updated last year
• apache / kyuubi

  View on GitHub
  Apache Kyuubi is a distributed and multi-tenant gateway to provide serverless SQL on data warehouses and lakehouses.
  ☆2,300Feb 6, 2026Updated last week