This project showcases how to integrate the world of DevOps, focusing on Continuous Integration (CI) and Continuous Deployment (CD) with the realm of modern data engineering using Terraform and Azure as the case study
☆14Dec 27, 2023Updated 2 years ago
Alternatives and similar repositories for cicd_for_data_engineering
Users that are interested in cicd_for_data_engineering are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- In this project, we setup and end to end data engineering using Apache Spark, Azure Databricks, Data Build Tool (DBT) using Azure as our …☆40Dec 18, 2023Updated 2 years ago
- This repository contains the necessary configuration files and DAGs (Directed Acyclic Graphs) for setting up a robust data engineering en…☆25Jan 26, 2024Updated 2 years ago
- This project provides an end-to-end data processing and visualization of visa numbers in Japan using PySpark and Plotly. The spark cluste…☆11Oct 11, 2023Updated 3 years ago
- This repository contains an end-to-end data engineering project using Apache Flink, focused on performing sales analytics. The project de…☆12Nov 18, 2023Updated 3 years ago
- This project serves as a comprehensive guide to building an end-to-end data engineering pipeline using TCP/IP Socket, Apache Spark, OpenA…☆47Jan 4, 2024Updated 2 years ago
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- An end-to-end data engineering pipeline that fetches real-time YouTube analytics and streams them through Kafka for processing with ksqlD…☆16Sep 19, 2023Updated 3 years ago
- This repository contains the code for a realtime election voting system. The system is built using Python, Kafka, Spark Streaming, Postgr…☆50Dec 11, 2023Updated 2 years ago
- ☆13Dec 30, 2022Updated 3 years ago
- This demo shows how to stream data to cloud databases with Confluent. It includes fully-managed connectors (Oracle CDC, RabbitMQ, MongoDB…☆11Jan 10, 2025Updated last year
- Azure Data Engineer Project☆10Jan 15, 2024Updated 2 years ago
- Real World Project on Formula1 Racing using Azure Databricks, Delta Lake and Azure Data Factory☆13Jul 24, 2023Updated 3 years ago
- This project shows how to capture changes from postgres database and stream them into kafka☆44May 17, 2024Updated 2 years ago
- ☆10Oct 9, 2021Updated 5 years ago
- ☆10Sep 9, 2021Updated 5 years ago
- Open source password manager - Proton Pass • AdSecurely store, share, and autofill your credentials with Proton Pass, the end-to-end encrypted password manager trusted by millions.
- ☆14Oct 17, 2023Updated 3 years ago
- This project demonstrates how to use Apache Airflow to submit jobs to Apache spark cluster in different programming laguages using Python…☆47Mar 14, 2024Updated 2 years ago
- Repo to show how to modularize messy SQL☆10Feb 7, 2025Updated last year
- ☆11Sep 16, 2021Updated 5 years ago
- ☆12Apr 17, 2023Updated 3 years ago
- This is a simple iris flower classification model deployment project as flask app on Docker or Kubernetes.☆11Feb 16, 2022Updated 4 years ago
- This repository contains an Apache Flink application for real-time sales analytics built using Docker Compose to orchestrate the necessar…☆53Dec 4, 2023Updated 2 years ago
- An end-to-end data engineering pipeline that fetches data from Wikipedia, cleans and transforms it with Apache Airflow and saves it on Az…☆32Oct 2, 2023Updated 3 years ago
- ☆14Nov 10, 2022Updated 4 years ago
- Deploy open-source AI quickly and easily - Special Bonus Offer • AdRunpod Hub is built for open source. One-click deployment and autoscaling endpoints without provisioning your own infrastructure.
- Example of how to build machine learning training workflow on AWS by Prefect☆12Nov 2, 2022Updated 4 years ago
- Undefined yet.☆10Jan 5, 2023Updated 3 years ago
- Automation of Databricks workflows☆13Nov 9, 2025Updated last year
- Demoing how to use Azure Key Vault Secrets from within Azure DevOps Pipelines.☆16Updated this week
- An end-to-end data engineering pipeline that orchestrates data ingestion, processing, and storage using Apache Airflow, Python, Apache Ka…☆381Feb 14, 2025Updated last year
- ☆20Jun 1, 2025Updated last year
- List of Computer Science courses with video lectures.☆12Mar 20, 2023Updated 3 years ago
- ☆13Jun 14, 2024Updated 2 years ago
- ☆21Mar 17, 2025Updated last year
- Deploy open-source AI quickly and easily - Special Bonus Offer • AdRunpod Hub is built for open source. One-click deployment and autoscaling endpoints without provisioning your own infrastructure.
- Demoing how to use Matrix and Each definitions in Azure DevOps YAML pipelines.☆20Apr 1, 2026Updated 7 months ago
- Nishiki is an app for tracking and sharing food inventories within groups for better pantry management.☆23Nov 1, 2024Updated 2 years ago
- CS886: Graph Neural Networks☆14Mar 28, 2025Updated last year
- ☆14Jun 9, 2023Updated 3 years ago
- This repository contains valuable insights and visualizations derived from an extensive Pizza dataset with over 48,000 rows.☆18Aug 24, 2023Updated 3 years ago
- Updated OOP notes, Fall 2021☆14Mar 9, 2022Updated 4 years ago
- AI Reliability Engineering - ARIA☆14Aug 5, 2026Updated 3 months ago
Are these results useful?