delta-io / kafka-delta-ingest
A highly efficient daemon for streaming data from Kafka into Delta Lake
☆379Updated this week
Alternatives and similar repositories for kafka-delta-ingest:
Users that are interested in kafka-delta-ingest are comparing it to the libraries listed below
- Lakekeeper: A Rust native Iceberg REST Catalog☆377Updated this week
- A native Delta implementation for integration with any query engine☆174Updated this week
- Apache PyIceberg☆551Updated this week
- Apache DataFusion Comet Spark Accelerator☆866Updated this week
- Apache Spark Connect Client for Rust☆95Updated last week
- The Trino (https://trino.io/) adapter plugin for dbt (https://getdbt.com)☆219Updated 3 weeks ago
- Open Control Plane for Tables in Data Lakehouse☆321Updated this week
- Nessie: Transactional Catalog for Data Lakes with Git-like semantics☆1,101Updated this week
- An open protocol for secure data sharing☆791Updated 2 weeks ago
- Snowflake Data Source for Apache Spark.☆222Updated last month
- dbt-spark contains all of the code enabling dbt to work with Apache Spark and Databricks☆413Updated this week
- A Python Library to support running data quality rules while the spark job is running⚡☆167Updated last week
- The native Rust implementation for Apache Hudi, with Python API bindings.☆185Updated this week
- Schema modelling framework for decentralised domain-driven ownership of data.☆249Updated last year
- Open, Multi-modal Catalog for Data & AI, written in Rust☆76Updated 3 months ago
- Rust implementation of Apache Iceberg with integration for Datafusion☆133Updated this week
- Delta Lake helper methods in PySpark☆312Updated 4 months ago
- Low Cost, Simple and Scalable Way of Data Replication to Apache Iceberg/Cloud/Data Lake☆215Updated this week
- ☆256Updated 2 months ago
- Apache Iceberg☆778Updated this week
- Serverless HTAP cloud data platform powered by Arrow × DuckDB × Iceberg☆315Updated last year
- Avro SerDe for Apache Spark structured APIs.☆231Updated 5 months ago
- A library that provides useful extensions to Apache Spark and PySpark.☆205Updated last month
- The Internals of Delta Lake☆183Updated this week
- ☆191Updated last week