A curated list of open source tools used in analytics platforms and data engineering ecosystem
☆588Mar 12, 2025Updated last year
Alternatives and similar repositories for awesome-open-source-data-engineering
Users that are interested in awesome-open-source-data-engineering are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- An Awesome List of Open-Source Data Engineering Projects☆3,247Oct 4, 2024Updated last year
- Scalable and efficient data transformation framework - backwards compatible with dbt.☆3,208Updated this week
- This is a repo with links to everything you'd ever want to learn about data engineering☆42,313Apr 2, 2026Updated 3 months ago
- Nessie: Transactional Catalog for Data Lakes with Git-like semantics☆1,481Updated this week
- A curated list of data engineering tools for software developers☆8,876Updated this week
- Virtual machines for every use case on DigitalOcean • AdGet dependable uptime with 99.99% SLA, simple security tools, and predictable monthly pricing with DigitalOcean's virtual machines, called Droplets.
- Code for DE101 book at https://de101.startdataengineering.com/☆119Feb 22, 2026Updated 4 months ago
- Ecommerce Realtime Data Pipeline (Data Modeling, Workflow Orchestration, Change Data Capture, Analytical Database and Dashboarding)☆71Mar 9, 2024Updated 2 years ago
- demo examples how to load data from different sources to different destinations☆31Jun 22, 2026Updated 3 weeks ago
- data load tool (dlt) is an open source Python library that makes data loading easy 🛠️☆5,632Updated this week
- 📙 Awesome Data Catalogs and Observability Platforms.☆1,052Aug 14, 2025Updated 11 months ago
- This course is designed to provide learners with the fundamental skills needed for data engineering using Python. The objective is to int…☆38Aug 15, 2024Updated last year
- Data Engineering Practice Problems☆2,791Jan 8, 2025Updated last year
- A curated list of awesome dbt resources☆1,706May 12, 2026Updated 2 months ago
- dlt-dagster-demo☆14Nov 6, 2023Updated 2 years ago
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- The best place to learn data engineering. Built and maintained by the data engineering community.☆2,004May 28, 2026Updated last month
- A super-minimal Docker Compose template for Apache Superset.☆18Jan 31, 2024Updated 2 years ago
- Apache Polaris, the interoperable, open source catalog for Apache Iceberg☆2,018Updated this week
- Lakekeeper is an Apache-Licensed, secure, fast and easy to use Apache Iceberg REST Catalog written in Rust.☆1,392Updated this week
- Roadmap for Data Engineering☆249Jul 1, 2026Updated 2 weeks ago
- Next generation compute platform for the post-modern data stack☆36Updated this week
- PyJaws: A Pythonic Way to Define Databricks Jobs and Workflows☆45Jan 24, 2026Updated 5 months ago
- Open, Multi-modal Catalog for Data & AI☆3,462Updated this week
- This project sets up a real-time data pipeline utilizing Change Data Capture (CDC) to stream changes from a PostgreSQL database to a Clic…☆12May 9, 2024Updated 2 years ago
- Wordpress hosting with auto-scaling - Free Trial Offer • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- open source data lake☆32Jan 17, 2025Updated last year
- An end-to-end open-source data stack for crawling and visualizing real estate data, facilitating insights into market trends.☆15Mar 16, 2026Updated 4 months ago
- A curated list of awesome DataOps tools☆234Dec 10, 2025Updated 7 months ago
- ☆267Oct 23, 2024Updated last year
- ☆69May 9, 2025Updated last year
- Open-source SQL workbench powered by DuckDB. Query local files, BigQuery, and Snowflake right in your browser. Drop-in UI for duckdb -ui.…☆79Jun 8, 2026Updated last month
- SwiftLake: Java SQL engine built on Apache Iceberg and DuckDB for efficient lakehouse reads and writes☆33Aug 13, 2025Updated 11 months ago
- Here I will be exploring various tools and methods that are used in data engineering process with Python.☆21Jan 4, 2021Updated 5 years ago
- ☆64Updated this week
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- Slow & local data allows you to move fast and deliver business value for the 99.9% of the data challenges.☆396Sep 30, 2025Updated 9 months ago
- Business intelligence as code: build fast, interactive data visualizations in SQL and markdown☆6,716Feb 18, 2026Updated 5 months ago
- High-performance data engine for AI and multimodal workloads. Process images, audio, video, and structured data at any scale☆5,644Updated this week
- Terraform module to create AWS EMR resources 🇺🇦☆26Mar 11, 2026Updated 4 months ago
- Build data pipelines with SQL and Python, ingest data from different sources, add quality checks, and build end-to-end flows.☆1,646Updated this week
- An e2e pipeline using dlt, dagster, duckdb, and dbt-core☆23Mar 27, 2026Updated 3 months ago
- OptimizeQL — AI-powered SQL query optimizer☆30Mar 28, 2026Updated 3 months ago