delta-io / kafka-delta-ingest
A highly efficient daemon for streaming data from Kafka into Delta Lake
☆394Updated this week
Alternatives and similar repositories for kafka-delta-ingest:
Users that are interested in kafka-delta-ingest are comparing it to the libraries listed below
- Lakekeeper is an Apache-Licensed, secure, fast and easy to use Apache Iceberg REST Catalog written in Rust.☆582Updated this week
- A native Delta implementation for integration with any query engine☆221Updated this week
- The Trino (https://trino.io/) adapter plugin for dbt (https://getdbt.com)☆232Updated 3 weeks ago
- Nessie: Transactional Catalog for Data Lakes with Git-like semantics☆1,185Updated this week
- Apache DataFusion Comet Spark Accelerator☆935Updated this week
- Low Cost, Simple and Scalable Way of Data Replication to Apache Iceberg/Cloud/Data Lake☆246Updated this week
- A library that provides useful extensions to Apache Spark and PySpark.☆224Updated last month
- An open protocol for secure data sharing☆826Updated this week
- Snowflake Data Source for Apache Spark.☆223Updated 2 weeks ago
- Open Control Plane for Tables in Data Lakehouse☆341Updated 2 weeks ago
- ☆263Updated 6 months ago
- Schema modelling framework for decentralised domain-driven ownership of data.☆252Updated last year
- The native Rust implementation for Apache Hudi, with Python API bindings.☆209Updated this week
- Delta Lake helper methods in PySpark☆322Updated 7 months ago
- Apache DataFusion Python Bindings☆441Updated this week
- A Spark UI and Spark History Server alternative with CPU and Memory metrics! Delight is free, cross-platform, and open-source.☆345Updated 10 months ago
- Apache Spark Connect Client for Rust☆106Updated 2 months ago
- ☆256Updated last week
- Apache Iceberg☆894Updated this week
- Rust implementation of Apache Iceberg with integration for Datafusion☆166Updated 2 weeks ago
- CLI tool to bulk migrate the tables from one catalog another without a data copy☆77Updated last week
- Spark style guide☆259Updated 6 months ago
- A Python Library to support running data quality rules while the spark job is running⚡☆183Updated last week
- Open, Multi-modal Catalog for Data & AI, written in Rust☆78Updated 6 months ago
- Adapter for dbt that executes dbt pipelines on Apache Flink☆95Updated last year
- Apache PyIceberg☆687Updated this week
- dbt-spark contains all of the code enabling dbt to work with Apache Spark and Databricks☆426Updated 2 months ago
- Work with your web service, database, and streaming schemas in a single format.☆343Updated 3 weeks ago
- The Internals of Delta Lake☆183Updated 3 months ago
- Astro SDK allows rapid and clean development of {Extract, Load, Transform} workflows using Python and SQL, powered by Apache Airflow.☆369Updated this week