airscholar / cicd_for_data_engineeringLinks
This project showcases how to integrate the world of DevOps, focusing on Continuous Integration (CI) and Continuous Deployment (CD) with the realm of modern data engineering using Terraform and Azure as the case study
☆14Updated last year
Alternatives and similar repositories for cicd_for_data_engineering
Users that are interested in cicd_for_data_engineering are comparing it to the libraries listed below
Sorting:
- In this project, we setup and end to end data engineering using Apache Spark, Azure Databricks, Data Build Tool (DBT) using Azure as our …☆37Updated last year
- This project shows how to capture changes from postgres database and stream them into kafka☆38Updated last year
- This repository contains the code for a realtime election voting system. The system is built using Python, Kafka, Spark Streaming, Postgr…☆43Updated 2 years ago
- Git Repo for EDW Best Practice Assets on the Lakehouse☆16Updated 2 years ago
- ☆19Updated last year
- This project serves as a comprehensive guide to building an end-to-end data engineering pipeline using TCP/IP Socket, Apache Spark, OpenA…☆43Updated last year
- This repository contains an end-to-end data engineering project using Apache Flink, focused on performing sales analytics. The project de…☆11Updated 2 years ago
- This repository contains the necessary configuration files and DAGs (Directed Acyclic Graphs) for setting up a robust data engineering en…☆23Updated last year
- A custom end-to-end analytics platform for customer churn☆11Updated 6 months ago
- Code to demonstrate data engineering metadata & logging best practices☆17Updated last year
- Deploy a complete data stack in just a couple of minutes.☆15Updated last year
- The resources of the preparation course for Databricks Data Engineer Professional certification exam☆158Updated 2 weeks ago
- ☆55Updated last year
- Example repo to create end to end tests for data pipeline.☆25Updated last year
- Code for blog at: https://www.startdataengineering.com/post/docker-for-de/☆40Updated last year
- Code snippets for Data Engineering Design Patterns book☆288Updated 8 months ago
- This project leverages GCS, Composer, Dataflow, BigQuery, and Looker on Google Cloud Platform (GCP) to build a robust data engineering so…☆31Updated last year
- End to end data engineering project☆57Updated 3 years ago
- ☆117Updated last year
- Sample project to demonstrate data engineering best practices☆201Updated last year
- Ultimate guide for mastering Spark Performance Tuning and Optimization concepts and for preparing for Data Engineering interviews☆177Updated 3 months ago
- Sample repo for startdataengineering DE 101 free course☆71Updated last year
- Notebooks to learn Databricks Lakehouse Platform☆38Updated last week
- ☆27Updated 7 months ago
- This repo contains "Databricks Certified Data Engineer Professional" Questions and related docs.☆120Updated last year
- Realtime Data Engineering Project☆30Updated 10 months ago
- My notes for AWS Data Engineer Associate☆97Updated last year
- With everything I learned from DEZoomcamp from datatalks.club, this project performs a batch processing on AWS for the cycling dataset wh…☆14Updated 3 years ago
- This project demonstrates how to use Apache Airflow to submit jobs to Apache spark cluster in different programming laguages using Python…☆46Updated last year
- Project for "Data pipeline design patterns" blog.☆47Updated last year