airscholar / cicd_for_data_engineeringLinks
This project showcases how to integrate the world of DevOps, focusing on Continuous Integration (CI) and Continuous Deployment (CD) with the realm of modern data engineering using Terraform and Azure as the case study
☆15Updated 2 years ago
Alternatives and similar repositories for cicd_for_data_engineering
Users that are interested in cicd_for_data_engineering are comparing it to the libraries listed below
Sorting:
- In this project, we setup and end to end data engineering using Apache Spark, Azure Databricks, Data Build Tool (DBT) using Azure as our …☆38Updated 2 years ago
- This project shows how to capture changes from postgres database and stream them into kafka☆40Updated last year
- Git Repo for EDW Best Practice Assets on the Lakehouse☆16Updated 2 years ago
- This repository contains the code for a realtime election voting system. The system is built using Python, Kafka, Spark Streaming, Postgr…☆45Updated 2 years ago
- ☆19Updated 2 years ago
- This repository contains an end-to-end data engineering project using Apache Flink, focused on performing sales analytics. The project de…☆11Updated 2 years ago
- This project serves as a comprehensive guide to building an end-to-end data engineering pipeline using TCP/IP Socket, Apache Spark, OpenA…☆44Updated 2 years ago
- The resources of the preparation course for Databricks Data Engineer Professional certification exam☆172Updated last month
- Code snippets for Data Engineering Design Patterns book☆331Updated last month
- A custom end-to-end analytics platform for customer churn☆11Updated 8 months ago
- An end-to-end data engineering pipeline that orchestrates data ingestion, processing, and storage using Apache Airflow, Python, Apache Ka…☆312Updated 11 months ago
- This repository contains the necessary configuration files and DAGs (Directed Acyclic Graphs) for setting up a robust data engineering en…☆23Updated 2 years ago
- Example repo to create end to end tests for data pipeline.☆25Updated last year
- Companion repository for the "Streamlining AWS Glue CI/CD — A Comprehensive Blueprint" blog post☆11Updated last year
- Project for "Data pipeline design patterns" blog.☆50Updated last year
- Code for blog at: https://www.startdataengineering.com/post/docker-for-de/☆40Updated last year
- ☆126Updated last year
- ☆59Updated 2 years ago
- Local Environment to Practice Data Engineering☆144Updated last year
- GitHub repository related to the course Mastering Elastic Map Reduce for Data Engineers☆24Updated 3 years ago
- ☆70Updated this week
- Code to demonstrate data engineering metadata & logging best practices☆20Updated last year
- End to end data engineering project☆58Updated 3 years ago
- End-to-end data platform leveraging the Modern data stack☆52Updated last year
- Ultimate guide for mastering Spark Performance Tuning and Optimization concepts and for preparing for Data Engineering interviews☆200Updated last month
- Open source stack lakehouse☆25Updated last year
- Notebooks to learn Databricks Lakehouse Platform☆40Updated this week
- My Setup Development Environment as Data Engineer☆35Updated 6 months ago
- Deploy a complete data stack in just a couple of minutes.☆15Updated last year
- Realtime Data Engineering Project☆30Updated last year