dipankarmazumdar / awesome-lakehouse-guide
Repo for everything open table formats (Iceberg, Hudi, Delta Lake) and the overall Lakehouse architecture
☆28Updated 2 weeks ago
Related projects ⓘ
Alternatives and complementary repositories for awesome-lakehouse-guide
- A Table format agnostic data sharing framework☆38Updated 9 months ago
- A Python Library to support running data quality rules while the spark job is running⚡☆162Updated this week
- Yet Another (Spark) ETL Framework☆18Updated last year
- Delta Lake Documentation☆46Updated 4 months ago
- CLI tool to bulk migrate the tables from one catalog another without a data copy☆61Updated this week
- Code snippets for Data Engineering Design Patterns book☆38Updated this week
- Unity Catalog UI☆39Updated 2 months ago
- ☆25Updated 6 months ago
- A tool to validate data, built around Apache Spark.☆101Updated this week
- Generate authentic looking mock data based on a SQL, JSON or Avro schema and produce to Kafka in JSON or Avro format.☆144Updated this week
- Quick Guides from Dremio on Several topics☆63Updated 2 weeks ago
- Docker envinroment to stream data from Kafka to Iceberg tables☆24Updated 8 months ago
- Library to convert DBT manifest metadata to Airflow tasks☆46Updated 8 months ago
- 📚 Tech blogs & talks by companies that run Apache Flink in production☆156Updated this week
- Don't Panic. This guide will help you when it feels like the end of the world.☆20Updated 5 months ago
- Delta Lake helper methods. No Spark dependency.☆22Updated 2 months ago
- Code snippets used in demos recorded for the blog.☆29Updated 3 weeks ago
- ☆252Updated 3 weeks ago
- Performance Observability for Apache Spark☆196Updated last week
- Flowchart for debugging Spark applications☆101Updated last month
- Adapter for dbt that executes dbt pipelines on Apache Flink☆84Updated 7 months ago
- Delta reader for the Ray open-source toolkit for building ML applications☆42Updated 9 months ago
- Multi-hop declarative data pipelines☆91Updated last week
- ☆47Updated 7 months ago
- Spark-Dashboard is a solution for monitoring Apache Spark jobs. This repository provides the tooling and configuration for deploying an A…☆111Updated 3 months ago
- Utility functions for dbt projects running on Trino☆20Updated 11 months ago
- Pythonic Programming Framework to orchestrate jobs in Databricks Workflow☆187Updated last week
- PyJaws: A Pythonic Way to Define Databricks Jobs and Workflows☆41Updated 4 months ago
- Delta Lake examples☆205Updated last month