qwshen / spark-flight-connector
A Spark Connector that reads data from / writes data to Arrow-Flight end-points with Arrow-Flight and Flight-SQL
☆37Updated 2 months ago
Related projects ⓘ
Alternatives and complementary repositories for spark-flight-connector
- ☆104Updated last year
- Implements a gateway that speaks the SparkConnect protocol and drives a backend using Substrait (over ADBC Flight SQL).☆16Updated last month
- ☆77Updated this week
- ☆31Updated this week
- CLI tool to bulk migrate the tables from one catalog another without a data copy☆61Updated this week
- ☆33Updated last year
- An Extensible Data Skipping Framework☆42Updated last year
- Rust implementation of Apache Iceberg with integration for Datafusion☆107Updated this week
- Code and examples of how to write and deploy Apache Spark Plugins. Spark plugins allow runnig custom code on the executors as they are in…☆85Updated 7 months ago
- Spark-Radiant is Apache Spark Performance and Cost Optimizer☆25Updated 2 years ago
- Lakekeeper: A Rust native Iceberg REST Catalog☆234Updated this week
- Gluten: Plugin to Boost Trino's Performance☆70Updated last year
- Point-in-Time optimizations for Apache Spark☆29Updated 10 months ago
- Apache DataFusion Ray☆116Updated this week
- Apache Paimon Rust The rust implementation of Apache Paimon.☆100Updated last month
- LST-Bench is a framework that allows users to run benchmarks specifically designed for evaluating Log-Structured Tables (LSTs) such as De…☆69Updated 2 weeks ago
- A native Delta implementation for integration with any query engine☆144Updated this week
- Java binding to Apache DataFusion☆70Updated this week
- Open, Multi-modal Catalog for Data & AI, written in Rust☆74Updated last month
- Spark* shuffle plugin for support shuffling data through a remote Hadoop-compatible file system, as opposed to vanilla Spark's local-dis…☆21Updated 8 months ago
- A re-implementation of Hadoop DistCP in Apache Spark☆44Updated 11 months ago
- Replicates any database (CDC events) to Apache Iceberg (To Cloud Storage)☆199Updated last week
- A native Rust library for Apache Hudi, with bindings into Python☆146Updated this week
- Spark* plug-in for accelerating Spark* SQL performance by using cache and index at SQL data source layer.☆37Updated last year
- Visualize column-level data lineage in Spark SQL☆87Updated 2 years ago
- ☆67Updated 2 weeks ago
- Apache datasketches☆88Updated last year
- ☆159Updated last month
- Apache Spark Kubernetes Operator☆65Updated last week