The Lakehouse Engine is a configuration driven Spark framework, written in Python, serving as a scalable and distributed engine for several lakehouse algorithms, data flows and utilities for Data Products.
☆293Oct 30, 2026Updated last week
Alternatives and similar repositories for lakehouse-engine
Users that are interested in lakehouse-engine are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Pythonic Programming Framework to orchestrate jobs in Databricks Workflow☆227Aug 12, 2026Updated 2 months ago
- Metadata driven Spark Declarative Pipelines framework for bronze/silver pipelines☆294Updated this week
- Delta Lake helper methods in PySpark☆329Jan 19, 2026Updated 9 months ago
- Generate relevant synthetic data quickly for your projects. The Databricks Labs synthetic data generator (aka `dbldatagen`) may be used …☆505Oct 26, 2026Updated last week
- A Python Library to support running data quality rules while the spark job is running⚡☆203Oct 7, 2026Updated 3 weeks ago
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- Yet Another (Spark) ETL Framework☆21Oct 21, 2023Updated 3 years ago
- Databricks framework to validate Data Quality of pySpark DataFrames and Tables☆479Updated this week
- ☆49Oct 15, 2024Updated 2 years ago
- ☆17Aug 28, 2025Updated last year
- This repo is a collection of tools to deploy, manage and operate a Databricks based Lakehouse.☆45Jan 27, 2025Updated last year
- Meta data driven spark notebooks, for loading data in Microsoft Fabric☆13Jul 21, 2024Updated 2 years ago
- PySpark test helper methods with beautiful error messages☆777Oct 29, 2026Updated last week
- Open, Multi-modal Catalog for Data & AI☆3,571Updated this week
- A DataOps framework for building a lakehouse.☆57Oct 27, 2026Updated last week
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- ☆13Mar 7, 2025Updated last year
- M3D Engine is a Spark application for the development of scalable data transformations and ingestions in data lakes.☆19May 4, 2021Updated 5 years ago
- ☆72Updated this week
- ☆18Aug 6, 2024Updated 2 years ago
- ☆25Mar 15, 2024Updated 2 years ago
- ☆50May 14, 2025Updated last year
- ☆10Nov 17, 2021Updated 4 years ago
- Possibly the fastest DataFrame-agnostic quality check library in town.☆250Feb 5, 2026Updated 9 months ago
- Demos to implement your Databricks Lakehouse☆448Oct 26, 2026Updated last week
- Proton VPN Special Offer - Get 70% off • AdSpecial partner offer. Trusted by over 100 million users worldwide. Tested, Approved and Recommended by Experts.
- Fabric Python Notebooks examples☆111Sep 1, 2026Updated 2 months ago
- Accelerates migrations to Databricks by automating key migration activities☆168Oct 30, 2026Updated last week
- Set of Terraform automation templates and quickstart demos to jumpstart the design of a Lakehouse on Databricks. This project has incorpo…☆92Jan 25, 2024Updated 2 years ago
- Python Testing for Databricks☆144Updated this week
- ☆17May 26, 2025Updated last year
- A dbt adapter for Databricks.☆389Updated this week
- Ready-to-use code snippets for building interactive Databricks Apps.☆187Oct 21, 2026Updated 2 weeks ago
- Code to help generate SQL for stakeholders. Code at https://www.startdataengineering.com/post/data-democratize-llm/☆13May 24, 2024Updated 2 years ago
- The Metadata Driven framework for Databricks Lakeflow Declarative Pipelines (formerly Delta Live Tables). Metadata framework that generat…☆67Updated this week
- Managed Kubernetes at scale on DigitalOcean • AdDigitalOcean Kubernetes includes the control plane, bandwidth allowance, container registry, automatic updates, and more for free.
- A Table format agnostic data sharing framework☆42Feb 4, 2024Updated 2 years ago
- ☆19Oct 26, 2021Updated 5 years ago
- Super Mario Demo Project☆10Dec 14, 2023Updated 2 years ago
- Complete data engineering pipeline running on Minikube Kubernetes, Argo CD, Spark, Trino, S3, Delta lake, Postgres+ Debezium CDC, MySQL,…☆28May 19, 2025Updated last year
- Data pipeline project using Data Factory, Databricks and Cosmosdb Graph, deployed using Azure DevOps, secured using firewalls and Azure A…☆11Dec 14, 2022Updated 3 years ago
- Databricks SDK for Python (Beta)☆577Updated this week
- A Fabric notebook that extracts the latest official Fabric SVG icons☆18Jun 28, 2026Updated 4 months ago