airscholar / cicd_for_data_engineeringLinks
This project showcases how to integrate the world of DevOps, focusing on Continuous Integration (CI) and Continuous Deployment (CD) with the realm of modern data engineering using Terraform and Azure as the case study
☆15Updated 2 years ago
Alternatives and similar repositories for cicd_for_data_engineering
Users that are interested in cicd_for_data_engineering are comparing it to the libraries listed below
Sorting:
- In this project, we setup and end to end data engineering using Apache Spark, Azure Databricks, Data Build Tool (DBT) using Azure as our …☆38Updated 2 years ago
- This project shows how to capture changes from postgres database and stream them into kafka☆40Updated last year
- This repository contains the code for a realtime election voting system. The system is built using Python, Kafka, Spark Streaming, Postgr…☆45Updated 2 years ago
- This project serves as a comprehensive guide to building an end-to-end data engineering pipeline using TCP/IP Socket, Apache Spark, OpenA…☆44Updated 2 years ago
- Git Repo for EDW Best Practice Assets on the Lakehouse☆16Updated 2 years ago
- An end-to-end data engineering pipeline that orchestrates data ingestion, processing, and storage using Apache Airflow, Python, Apache Ka…☆309Updated 11 months ago
- A custom end-to-end analytics platform for customer churn☆11Updated 8 months ago
- Simple stream processing pipeline☆110Updated last year
- This repository contains the necessary configuration files and DAGs (Directed Acyclic Graphs) for setting up a robust data engineering en…☆23Updated 2 years ago
- This repository contains an end-to-end data engineering project using Apache Flink, focused on performing sales analytics. The project de…☆11Updated 2 years ago
- Code snippets for Data Engineering Design Patterns book☆317Updated 3 weeks ago
- Code for blog at: https://www.startdataengineering.com/post/docker-for-de/☆40Updated last year
- End to end data engineering project☆58Updated 3 years ago
- ☆56Updated 2 years ago
- ☆19Updated 2 years ago
- The resources of the preparation course for Databricks Data Engineer Professional certification exam☆170Updated last month
- Code to demonstrate data engineering metadata & logging best practices☆20Updated last year
- Sample project to demonstrate data engineering best practices☆202Updated last year
- ☆70Updated last week
- ☆16Updated last year
- Stream processing pipeline from Finnhub websocket using Spark, Kafka, Kubernetes and more☆373Updated 2 years ago
- This repo contains "Databricks Certified Data Engineer Professional" Questions and related docs.☆131Updated last year
- This project leverages GCS, Composer, Dataflow, BigQuery, and Looker on Google Cloud Platform (GCP) to build a robust data engineering so…☆33Updated 2 years ago
- Code for "Advanced data transformations in SQL" free live workshop☆89Updated 8 months ago
- ☆45Updated last year
- GitHub repository related to the course Mastering Elastic Map Reduce for Data Engineers☆24Updated 3 years ago
- End-to-end data platform leveraging the Modern data stack☆52Updated last year
- Code for dbt tutorial☆166Updated 4 months ago
- Local Environment to Practice Data Engineering☆143Updated last year
- Code for blog at https://www.startdataengineering.com/post/python-for-de/☆98Updated last year