A highly efficient daemon for streaming data from Kafka into Delta Lake
☆428May 5, 2025Updated 10 months ago
Alternatives and similar repositories for kafka-delta-ingest
Users that are interested in kafka-delta-ingest are comparing it to the libraries listed below
Sorting:
- A native Rust library for Delta Lake, with bindings into Python☆3,160Mar 2, 2026Updated last week
- A native Delta implementation for integration with any query engine☆319Updated this week
- An open protocol for secure data sharing☆920Mar 3, 2026Updated last week
- Delta Lake helper methods in PySpark☆327Jan 19, 2026Updated last month
- An open-source storage framework that enables building a Lakehouse architecture with compute engines including Spark, PrestoDB, Flink, Tr…☆8,624Updated this week
- Apache DataFusion Comet Spark Accelerator☆1,150Updated this week
- Apache Iceberg☆1,234Updated this week
- Apache DataFusion Ballista Distributed Query Engine☆1,988Updated this week
- A Minimalistic Rust Implementation of Delta Sharing Server.☆98Mar 17, 2025Updated 11 months ago
- Apache Spark Connect Client for Rust☆117Jun 10, 2025Updated 9 months ago
- Delta reader for the Ray open-source toolkit for building ML applications☆45Jan 27, 2024Updated 2 years ago
- Open, Multi-modal Catalog for Data & AI, written in Rust☆86Sep 30, 2024Updated last year
- Apache DataFusion SQL Query Engine☆8,476Updated this week
- Nessie: Transactional Catalog for Data Lakes with Git-like semantics☆1,430Updated this week
- Lakekeeper is an Apache-Licensed, secure, fast and easy to use Apache Iceberg REST Catalog written in Rust.☆1,210Updated this week
- ☆89May 5, 2025Updated 10 months ago
- Open, Multi-modal Catalog for Data & AI☆3,327Updated this week
- Unofficial rust implementation of Apache Iceberg with integration for Datafusion☆233Feb 27, 2026Updated last week
- This repository has moved into https://github.com/dbt-labs/dbt-adapters☆444Jul 16, 2025Updated 7 months ago
- Streaming data changes to a Data Lake with Debezium and Delta Lake pipeline☆76Feb 15, 2023Updated 3 years ago
- LakeSail's computation framework with a mission to unify batch processing, stream processing, and compute-intensive AI workloads.☆1,181Updated this week
- Apache Polaris, the interoperable, open source catalog for Apache Iceberg☆1,866Updated this week
- A set of tools for writing servers that speak PostgreSQL's wire protocol☆93Feb 11, 2026Updated 3 weeks ago
- Distributed stream processing engine in Rust☆4,832Updated this week
- Low Cost, Simple and Scalable Way of Data Replication to Apache Iceberg/Cloud/Data Lake☆302Feb 23, 2026Updated 2 weeks ago
- A Python package to help Databricks Unity Catalog users to read and query Delta Lake tables with Polars, DuckDb, or PyArrow.☆27Mar 25, 2024Updated last year
- ☆81Apr 23, 2025Updated 10 months ago
- ☆14Feb 10, 2026Updated last month
- A proof-of-concept repo that attempts to use Apache Superset with a custom ADBC to Arrow Flight SQL SQLAlchemy driver.☆25Sep 8, 2023Updated 2 years ago
- An Open Standard for lineage metadata collection☆2,340Updated this week
- Demos for Nessie. Nessie provides Git-like capabilities for your Data Lake.☆30Updated this week
- High-performance data engine for AI and multimodal workloads. Process images, audio, video, and structured data at any scale☆5,268Mar 3, 2026Updated last week
- The Auron accelerator for distributed computing framework (e.g., Spark) leverages native vectorized execution to accelerate query process…☆1,719Updated this week
- Apache XTable (incubating) is a cross-table converter for lakehouse table formats that facilitates interoperability across data processin…☆1,164Feb 23, 2026Updated 2 weeks ago
- Delta Lake Documentation☆53Jun 19, 2024Updated last year
- Deequ is a library built on top of Apache Spark for defining "unit tests for data", which measure data quality in large datasets.☆3,594Feb 17, 2026Updated 3 weeks ago
- Python Stream Processing☆1,958Mar 27, 2025Updated 11 months ago
- 🌊 Continuously synchronize the systems where your data lives, to the systems where you _want_ it to live, with Estuary Flow. 🌊☆890Updated this week
- Robust data transformation tool using SQL☆22Dec 20, 2022Updated 3 years ago