This project shows how to capture changes from postgres database and stream them into kafka
☆42May 17, 2024Updated last year
Alternatives and similar repositories for changecapture-e2e
Users that are interested in changecapture-e2e are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- This repository contains an end-to-end data engineering project using Apache Flink, focused on performing sales analytics. The project de…☆11Nov 18, 2023Updated 2 years ago
- An end-to-end data engineering pipeline that fetches real-time YouTube analytics and streams them through Kafka for processing with ksqlD…☆16Sep 19, 2023Updated 2 years ago
- This repository contains the code for a realtime election voting system. The system is built using Python, Kafka, Spark Streaming, Postgr…☆47Dec 11, 2023Updated 2 years ago
- This project showcases how to integrate the world of DevOps, focusing on Continuous Integration (CI) and Continuous Deployment (CD) with …☆14Dec 27, 2023Updated 2 years ago
- This project serves as a comprehensive guide to building an end-to-end data engineering pipeline using TCP/IP Socket, Apache Spark, OpenA…☆44Jan 4, 2024Updated 2 years ago
- Wordpress hosting with auto-scaling on Cloudways • AdFully Managed hosting built for WordPress-powered businesses that need reliable, auto-scalable hosting. Cloudways SafeUpdates now available.
- This repository contains the necessary configuration files and DAGs (Directed Acyclic Graphs) for setting up a robust data engineering en…☆25Jan 26, 2024Updated 2 years ago
- An end-to-end data engineering pipeline that orchestrates data ingestion, processing, and storage using Apache Airflow, Python, Apache Ka…☆323Feb 14, 2025Updated last year
- This repository contains an Apache Flink application for real-time sales analytics built using Docker Compose to orchestrate the necessar…☆50Dec 4, 2023Updated 2 years ago
- In this project, we setup and end to end data engineering using Apache Spark, Azure Databricks, Data Build Tool (DBT) using Azure as our …☆39Dec 18, 2023Updated 2 years ago
- A Python package that creates fine-grained dbt tasks on Apache Airflow☆19Apr 25, 2024Updated last year
- An end-to-end data engineering pipeline that fetches data from Wikipedia, cleans and transforms it with Apache Airflow and saves it on Az…☆32Oct 2, 2023Updated 2 years ago
- Data Engineering Bootcamp☆31Aug 5, 2025Updated 8 months ago
- An end-to-end, containerized data pipeline for near-real-time user event analytics using Kafka, ClickHouse, Airflow, and PySpark. Made to…☆73Sep 12, 2025Updated 6 months ago
- Apache Airflow advanced functionalities examples☆21Mar 22, 2024Updated 2 years ago
- Simple, predictable pricing with DigitalOcean hosting • AdAlways know what you'll pay with monthly caps and flat pricing. Enterprise-grade infrastructure trusted by 600k+ customers.
- ☆13Oct 8, 2025Updated 6 months ago
- Data Engineering Project in GCP☆22Mar 29, 2023Updated 3 years ago
- Building a Data Pipeline with an Open Source Stack☆58Jun 27, 2025Updated 9 months ago
- This is an end to end MLOps system☆34Nov 27, 2025Updated 4 months ago
- Repository to host micro service implementation patterns.☆14Jun 25, 2025Updated 9 months ago
- Compare Naive Bayes, SVM, XGBoost, Bagging, AdaBoost, K-Nearest Neighbors, Random Forests for classification of Malaria Cells☆11Jun 5, 2019Updated 6 years ago
- Get data from API, run a scheduled script with Airflow, send data to Kafka and consume with Spark, then write to Cassandra☆146Jul 27, 2023Updated 2 years ago
- MCP proxy: tool aggregation, search, filtering, security☆19Jul 15, 2025Updated 8 months ago
- Video Content-Based Advertisement Recommendation Using Text Classification☆10Dec 9, 2023Updated 2 years ago
- DigitalOcean Gradient AI Platform • AdBuild production-ready AI agents using customizable tools or access multiple LLMs through a single endpoint. Create custom knowledge bases or connect external data.
- GPT-4o Powered Calorie Detecor☆18May 29, 2024Updated last year
- Analyzing Video Assistant Referee (VAR) decisions in the English Premier League (2019 - 2021)☆12Jun 1, 2021Updated 4 years ago
- This solution helps you deploy ETL processes and data storage resources to create an Insurance Lake using Amazon S3 buckets for storage, …☆34Mar 12, 2026Updated last month
- ☆10Jan 8, 2024Updated 2 years ago
- Produce Kafka messages, consume them and upload into Cassandra, MongoDB.☆43Sep 26, 2023Updated 2 years ago
- text2sql with modern LLMs (duckdb-nsql, SQLCoder etc ...)☆18Apr 13, 2024Updated last year
- Spark application to consume kafka events generated by a python producer.☆12Aug 7, 2021Updated 4 years ago
- ☆10Jan 18, 2024Updated 2 years ago
- A demonstration of an ELT (Extract, Load, Transform) pipeline☆31Feb 19, 2024Updated 2 years ago
- NordVPN Special Discount Offer • AdSave on top-rated NordVPN 1 or 2-year plans with secure browsing, privacy protection, and support for for all major platforms.
- Transparent sandbox for integration testing against AWS services. Test your infrastructure without changes to your Terraform files or you…☆12Oct 26, 2023Updated 2 years ago
- It is a assemble to include all Practice Projects about Big Data Topic, includes Hadoop, Spark, Spark Streaming and Kafka☆11Mar 7, 2019Updated 7 years ago
- Project for "Data pipeline design patterns" blog.☆51Aug 6, 2024Updated last year
- TTS utility☆12Aug 2, 2020Updated 5 years ago
- ☆14Mar 11, 2023Updated 3 years ago
- THREE JS 2D buttons and labels achorable library☆13Nov 29, 2016Updated 9 years ago
- Design pattern for orchestrating an incremental data ingestion pipeline using AWS Step Functions from an on premise location into an Amaz…☆29Jul 24, 2019Updated 6 years ago