Alternatives and similar repositories for spark

Users that are interested in spark are comparing it to the libraries listed below

• uber / RemoteShuffleService
  Remote shuffle service for Apache Spark to store shuffle data on remote servers.
  ☆336Updated 2 years ago
• oap-project / gazelle_plugin
  Native SQL Engine plugin for Spark SQL with vectorized SIMD optimizations.
  ☆258Updated 2 years ago
• IBM / spark-tpc-ds-performance-test
  Use the TPC-DS benchmark to test Spark SQL performance
  ☆183Updated 5 years ago
• maropu / spark-tpcds-datagen
  All the things about TPC-DS in Apache Spark
  ☆108Updated 2 years ago
• boostscale / velox4j
  Community Java bindings for https://github.com/facebookincubator/velox
  ☆36Updated this week
• alibaba / SparkCube
  SparkCube is an open-source project for extremely fast OLAP data analysis. SparkCube is an extension of Apache Spark.
  ☆136Updated 2 years ago
• oap-project / remote-shuffle
  Spark* shuffle plugin for support shuffling data through a remote Hadoop-compatible file system, as opposed to vanilla Spark's local-dis…
  ☆21Updated last year
• linkedin / transport
  A framework for writing performant user-defined functions (UDFs) that are portable across a variety of engines including Apache Spark, Ap…
  ☆302Updated last month
• MemVerge / splash
  Splash, a flexible Spark shuffle manager that supports user-defined storage backends for shuffle data storage and exchange
  ☆130Updated 11 months ago
• apache / uniffle
  Uniffle is a high performance, general purpose Remote Shuffle Service.
  ☆433Updated this week
• ververica / frocksdb
  ☆66Updated last year
• Tencent / Firestorm
  Firestorm is a Remote Shuffle Service, and provides the capability for Apache Spark and Apache Hadoop MapReduce applications to store shu…
  ☆256Updated 2 years ago
• Intel-bigdata / SSM
  Smart Storage Management for Big Data, a comprehensive hot/cold data optimized solution
  ☆142Updated 2 years ago
• microsoft / lst-bench
  LST-Bench is a framework that allows users to run benchmarks specifically designed for evaluating Log-Structured Tables (LSTs) such as De…
  ☆87Updated last month
• apache / tez
  Apache Tez
  ☆508Updated last week
• linkedin / dynamometer
  A tool for scale and performance testing of HDFS with a specific focus on the NameNode.
  ☆134Updated last year
• cloudera / impala-tpcds-kit
  TPC-DS Kit for Impala
  ☆171Updated last year
• apache / orc
  Apache ORC - the smallest, fastest columnar storage for Hadoop workloads
  ☆752Updated 3 weeks ago
• CoxAutomotiveDataSolutions / spark-distcp
  A re-implementation of Hadoop DistCP in Apache Spark
  ☆47Updated last year
• JoshRosen / hive
  Mirror of Apache Hive
  ☆32Updated 5 years ago
• oap-project / Gluten-Trino
  Gluten: Plugin to Boost Trino's Performance
  ☆76Updated 2 years ago
• dataArtisans / yahoo-streaming-benchmark
  An extension of Yahoo's Benchmarks
  ☆108Updated last year
• ExpediaGroup / waggle-dance
  Hive federation service. Enables disparate tables to be concurrently accessed across multiple Hive deployments.
  ☆284Updated 2 weeks ago
• nexmark / nexmark
  Benchmarks for queries over continuous data streams.
  ☆368Updated 11 months ago
• qubole / rubix
  Cache File System optimized for columnar formats and object stores
  ☆186Updated 3 years ago
• Mellanox / SparkRDMA
  This is archive of SparkRDMA project. The new repository with RDMA shuffle acceleration for Apache Spark is here: https://github.com/Nvid…
  ☆255Updated 6 years ago
• oap-project / sql-ds-cache
  Spark* plug-in for accelerating Spark* SQL performance by using cache and index at SQL data source layer.
  ☆37Updated 2 years ago
• spirom / spark-data-sources
  Developing Spark External Data Sources using the V2 API
  ☆48Updated 7 years ago
• liancheng / spear
  A playground for experimenting ideas that may apply to Spark SQL/Catalyst
  ☆141Updated 7 years ago
• realtime-storage-engine / flink-spillable-statebackend
  The preview version of a spillable state backend for Apache Flink
  ☆39Updated 4 years ago