airscholar / cicd_for_data_engineeringLinks
This project showcases how to integrate the world of DevOps, focusing on Continuous Integration (CI) and Continuous Deployment (CD) with the realm of modern data engineering using Terraform and Azure as the case study
☆14Updated 2 years ago
Alternatives and similar repositories for cicd_for_data_engineering
Users that are interested in cicd_for_data_engineering are comparing it to the libraries listed below
Sorting:
- In this project, we setup and end to end data engineering using Apache Spark, Azure Databricks, Data Build Tool (DBT) using Azure as our …☆37Updated 2 years ago
- This project shows how to capture changes from postgres database and stream them into kafka☆38Updated last year
- ☆18Updated 2 years ago
- This repository contains the code for a realtime election voting system. The system is built using Python, Kafka, Spark Streaming, Postgr…☆43Updated 2 years ago
- This repository contains an end-to-end data engineering project using Apache Flink, focused on performing sales analytics. The project de…☆11Updated 2 years ago
- This project serves as a comprehensive guide to building an end-to-end data engineering pipeline using TCP/IP Socket, Apache Spark, OpenA…☆43Updated last year
- A custom end-to-end analytics platform for customer churn☆11Updated 7 months ago
- An end-to-end data engineering pipeline that orchestrates data ingestion, processing, and storage using Apache Airflow, Python, Apache Ka…☆300Updated 10 months ago
- Git Repo for EDW Best Practice Assets on the Lakehouse☆16Updated 2 years ago
- Realtime Data Engineering Project☆30Updated 11 months ago
- ☆27Updated 8 months ago
- Code snippets for Data Engineering Design Patterns book☆302Updated 2 weeks ago
- Sample project to demonstrate data engineering best practices☆204Updated last year
- Code to demonstrate data engineering metadata & logging best practices☆18Updated last year
- Companion repository for the "Streamlining AWS Glue CI/CD — A Comprehensive Blueprint" blog post☆12Updated last year
- ☆56Updated last year
- Simple stream processing pipeline☆110Updated last year
- This repository contains the necessary configuration files and DAGs (Directed Acyclic Graphs) for setting up a robust data engineering en…☆23Updated last year
- The resources of the preparation course for Databricks Data Engineer Professional certification exam☆163Updated 2 weeks ago
- Example repo to create end to end tests for data pipeline.☆25Updated last year
- Code for blog at https://www.startdataengineering.com/post/python-for-de/☆92Updated last year
- Code for blog at: https://www.startdataengineering.com/post/docker-for-de/☆41Updated last year
- End to end data engineering project with kafka, airflow, spark, postgres and docker.☆107Updated 9 months ago
- End-to-end data platform leveraging the Modern data stack☆52Updated last year
- ☆15Updated 2 years ago
- End to end data engineering project☆57Updated 3 years ago
- AWS ETL Pipleine☆29Updated last year
- This project demonstrates how to use Apache Airflow to submit jobs to Apache spark cluster in different programming laguages using Python…☆46Updated last year
- A template repository to create a data project with IAC, CI/CD, Data migrations, & testing☆281Updated last year
- GitHub repository related to the course Mastering Elastic Map Reduce for Data Engineers☆23Updated 3 years ago