Full stack data engineering tools and infrastructure set-up
☆58Feb 13, 2021Updated 5 years ago
Alternatives and similar repositories for data-engineering-devops
Users that are interested in data-engineering-devops are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Practical Data Engineering: A Hands-On Real-Estate Project Guide☆833Jun 25, 2026Updated 4 months ago
- dlt-dagster-demo☆14Nov 6, 2023Updated 3 years ago
- ecommerce GCP Streaming pipeline ― Cloud Storage, Compute Engine, Pub/Sub, Dataflow, Apache Beam, BigQuery and Tableau; GCP Batch pipelin…☆10Mar 9, 2022Updated 4 years ago
- The Open-Source Enterprise Data Platform in a single Portal☆270Updated this week
- Open Data Stack Projects: Examples of End to End Data Engineering Projects☆92Jun 25, 2023Updated 3 years ago
- Managed Database hosting by DigitalOcean • AdPostgreSQL, MySQL, MongoDB, Kafka, Valkey, and OpenSearch available. Automatically scale up storage and focus on building your apps.
- A portable Datamart and Business Intelligence suite built with Docker, Dagster, dbt, DuckDB and Superset☆266Oct 26, 2026Updated 3 weeks ago
- Pipeline for Automated Updates of Kaggle's "Formula 2 Dataset"☆11Aug 15, 2023Updated 3 years ago
- Data Engineering reference architecture for Kubernetes-based data platforms using GitOps workflow patterns. Includes infrastructure confi…☆14May 14, 2025Updated last year
- ☆19Updated this week
- Project based learning for Data Engineering fundamentals.☆13Jan 15, 2021Updated 5 years ago
- Firefox extension that shows parquet schema when going over GCP cloud storage. Use DuckDB WASM☆11Jan 19, 2024Updated 2 years ago
- Testing Boring SL with DuckDB☆32Aug 18, 2025Updated last year
- Example repo to create end to end tests for data pipeline.☆25Jun 14, 2024Updated 2 years ago
- dagster scikit-learn pipeline example.☆46Mar 18, 2023Updated 3 years ago
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- My first attempt at a rough ETL pipeline; technologies include spark, GCS, prefect orchestration, and terraform☆14Oct 12, 2022Updated 4 years ago
- ☆13Oct 4, 2023Updated 3 years ago
- One-command deployment: Hetzner + Cloudflare Zero Trust + Docker☆32Updated this week
- Bigdata on Kubernetes, Published by Packt☆38Oct 1, 2024Updated 2 years ago
- Star Schema Benchmark dbgen☆11Aug 13, 2020Updated 6 years ago
- Simple ETL pipeline using Python☆29May 22, 2023Updated 3 years ago
- Code for DE101 book at https://de101.startdataengineering.com/☆138Feb 22, 2026Updated 8 months ago
- ☆10Aug 26, 2026Updated 2 months ago
- A simple Data Engineering solution for testing or education purposes. You only need to know SQL and Python to understand this project. Da…☆29Jul 2, 2022Updated 4 years ago
- Wordpress hosting with auto-scaling - Free Trial Offer • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- Events about the open source data stack☆13Apr 16, 2022Updated 4 years ago
- An example Data Vault 2.0 data warehouse modelling Microsoft's Northwind sample database.☆13Oct 18, 2023Updated 3 years ago
- Deploy a complete data stack in just a couple of minutes.☆15Mar 6, 2024Updated 2 years ago
- Business challenge that requires building a data platform for retailer data analytics.☆18Feb 19, 2023Updated 3 years ago
- A Rosé Pine Theme for Obsidian☆15Oct 21, 2026Updated 3 weeks ago
- Demos for Nessie. Nessie provides Git-like capabilities for your Data Lake.☆32Nov 6, 2026Updated last week
- A collection of data engineering projects: data modeling, ETL pipelines, data lakes, infrastructure configuration on AWS, data warehousin…☆15Apr 29, 2021Updated 5 years ago
- Local development environment for python data projects, with Docker☆22Dec 14, 2022Updated 3 years ago
- ☆15Apr 22, 2024Updated 2 years ago
- Wordpress hosting with auto-scaling - Free Trial Offer • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- Repository for code examples from my youtube channel and medium articles working with data in python on AWS☆30Feb 5, 2024Updated 2 years ago
- Building a Modern Data Lake with Minio, Spark, Airflow via Docker.☆23May 11, 2024Updated 2 years ago
- A Table format agnostic data sharing framework☆42Feb 4, 2024Updated 2 years ago
- ☆22Jul 14, 2020Updated 6 years ago
- Workshop on Machine Learning in Python☆19Mar 24, 2016Updated 10 years ago
- Playground for Lakehouse (Iceberg, Hudi, Spark, Flink, Trino, DBT, Airflow, Kafka, Debezium CDC)☆72Sep 23, 2023Updated 3 years ago
- end-to-end data engineering project to get insights from PyPi using python, duckdb, MotherDuck☆238Sep 15, 2026Updated 2 months ago
Are these results useful?