delta-io / kafka-delta-ingestView external linksLinks
A highly efficient daemon for streaming data from Kafka into Delta Lake
☆427May 5, 2025Updated 9 months ago
Alternatives and similar repositories for kafka-delta-ingest
Users that are interested in kafka-delta-ingest are comparing it to the libraries listed below
Sorting:
- A native Rust library for Delta Lake, with bindings into Python☆3,135Updated this week
- A native Delta implementation for integration with any query engine☆313Updated this week
- An open protocol for secure data sharing☆919Feb 6, 2026Updated last week
- Delta Lake helper methods in PySpark☆327Jan 19, 2026Updated 3 weeks ago
- Apache DataFusion Comet Spark Accelerator☆1,134Updated this week
- An open-source storage framework that enables building a Lakehouse architecture with compute engines including Spark, PrestoDB, Flink, Tr…☆8,590Updated this week
- Apache Iceberg☆1,220Updated this week
- Apache DataFusion Ballista Distributed Query Engine☆1,973Updated this week
- A Minimalistic Rust Implementation of Delta Sharing Server.☆96Mar 17, 2025Updated 11 months ago
- Apache Spark Connect Client for Rust☆118Jun 10, 2025Updated 8 months ago
- Delta reader for the Ray open-source toolkit for building ML applications☆45Jan 27, 2024Updated 2 years ago
- Open, Multi-modal Catalog for Data & AI, written in Rust☆86Sep 30, 2024Updated last year
- Apache DataFusion SQL Query Engine☆8,398Updated this week
- Nessie: Transactional Catalog for Data Lakes with Git-like semantics☆1,414Updated this week
- Lakekeeper is an Apache-Licensed, secure, fast and easy to use Apache Iceberg REST Catalog written in Rust.☆1,193Updated this week
- Open, Multi-modal Catalog for Data & AI☆3,305Updated this week
- Unofficial rust implementation of Apache Iceberg with integration for Datafusion☆233Feb 10, 2026Updated last week
- This repository has moved into https://github.com/dbt-labs/dbt-adapters☆444Jul 16, 2025Updated 7 months ago
- Streaming data changes to a Data Lake with Debezium and Delta Lake pipeline☆76Feb 15, 2023Updated 3 years ago
- LakeSail's computation framework with a mission to unify batch processing, stream processing, and compute-intensive AI workloads.☆1,166Updated this week
- Apache Polaris, the interoperable, open source catalog for Apache Iceberg☆1,834Updated this week
- A set of tools for writing servers that speak PostgreSQL's wire protocol☆93Updated this week
- Distributed stream processing engine in Rust☆4,810Updated this week
- Low Cost, Simple and Scalable Way of Data Replication to Apache Iceberg/Cloud/Data Lake☆299Feb 10, 2026Updated last week
- A Python package to help Databricks Unity Catalog users to read and query Delta Lake tables with Polars, DuckDb, or PyArrow.☆27Mar 25, 2024Updated last year
- ☆81Apr 23, 2025Updated 9 months ago
- ☆86May 5, 2025Updated 9 months ago
- A proof-of-concept repo that attempts to use Apache Superset with a custom ADBC to Arrow Flight SQL SQLAlchemy driver.☆25Sep 8, 2023Updated 2 years ago
- High-performance data engine for AI and multimodal workloads. Process images, audio, video, and structured data at any scale☆5,200Updated this week
- Demos for Nessie. Nessie provides Git-like capabilities for your Data Lake.☆30Feb 1, 2026Updated 2 weeks ago
- An Open Standard for lineage metadata collection☆2,316Updated this week
- The Auron accelerator for distributed computing framework (e.g., Spark) leverages native vectorized execution to accelerate query process…☆1,708Updated this week
- Apache XTable (incubating) is a cross-table converter for lakehouse table formats that facilitates interoperability across data processin…☆1,161Updated this week
- Delta Lake Documentation☆53Jun 19, 2024Updated last year
- Deequ is a library built on top of Apache Spark for defining "unit tests for data", which measure data quality in large datasets.☆3,580Feb 2, 2026Updated 2 weeks ago
- 🌊 Continuously synchronize the systems where your data lives, to the systems where you _want_ it to live, with Estuary Flow. 🌊☆876Updated this week
- Python Stream Processing☆1,958Mar 27, 2025Updated 10 months ago
- Robust data transformation tool using SQL☆22Dec 20, 2022Updated 3 years ago
- A User-Defined Function Framework for Apache Arrow.☆111Feb 1, 2026Updated 2 weeks ago