This project shows how to capture changes from postgres database and stream them into kafka
☆42May 17, 2024Updated 2 years ago
Alternatives and similar repositories for changecapture-e2e
Users that are interested in changecapture-e2e are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- This repository contains an end-to-end data engineering project using Apache Flink, focused on performing sales analytics. The project de…☆12Nov 18, 2023Updated 2 years ago
- An end-to-end data engineering pipeline that fetches real-time YouTube analytics and streams them through Kafka for processing with ksqlD…☆16Sep 19, 2023Updated 2 years ago
- This repository contains the code for a realtime election voting system. The system is built using Python, Kafka, Spark Streaming, Postgr…☆48Dec 11, 2023Updated 2 years ago
- This project showcases how to integrate the world of DevOps, focusing on Continuous Integration (CI) and Continuous Deployment (CD) with …☆14Dec 27, 2023Updated 2 years ago
- This project serves as a comprehensive guide to building an end-to-end data engineering pipeline using TCP/IP Socket, Apache Spark, OpenA…☆45Jan 4, 2024Updated 2 years ago
- Proton VPN Special Offer - Get 70% off • AdSpecial partner offer. Trusted by over 100 million users worldwide. Tested, Approved and Recommended by Experts.
- This project demonstrates how to use Apache Airflow to submit jobs to Apache spark cluster in different programming laguages using Python…☆48Mar 14, 2024Updated 2 years ago
- An end-to-end data engineering pipeline that orchestrates data ingestion, processing, and storage using Apache Airflow, Python, Apache Ka…☆331Feb 14, 2025Updated last year
- In this project, we setup and end to end data engineering using Apache Spark, Azure Databricks, Data Build Tool (DBT) using Azure as our …☆39Dec 18, 2023Updated 2 years ago
- This project provides a comprehensive data pipeline solution to extract, transform, and load (ETL) Reddit data into a Redshift data wareh…☆217Oct 23, 2023Updated 2 years ago
- An end-to-end data engineering pipeline that fetches data from Wikipedia, cleans and transforms it with Apache Airflow and saves it on Az…☆30Oct 2, 2023Updated 2 years ago
- Data Engineering Bootcamp☆33Aug 5, 2025Updated 10 months ago
- This construct builds some elements for you to quickly launch an EMR Serverless application. After submitting the Emr Serverless job, you…☆11Nov 18, 2025Updated 7 months ago
- An end-to-end, containerized data pipeline for near-real-time user event analytics using Kafka, ClickHouse, Airflow, and PySpark. Made to…☆80Sep 12, 2025Updated 9 months ago
- Apache Airflow advanced functionalities examples☆21Mar 22, 2024Updated 2 years ago
- Virtual machines for every use case on DigitalOcean • AdGet dependable uptime with 99.99% SLA, simple security tools, and predictable monthly pricing with DigitalOcean's virtual machines, called Droplets.
- velib-v2: An ETL pipeline that employs batch and streaming jobs using Spark, Kafka, Airflow, and other tools, all orchestrated with Docke…☆21Aug 12, 2025Updated 10 months ago
- Data Engineering Project in GCP☆22Mar 29, 2023Updated 3 years ago
- The Data Pipeline and Analytics Stack is a comprehensive solution designed for processing, storing, and visualizing data. Explore a compl…☆18Dec 26, 2023Updated 2 years ago
- will add all data science project that I'll do.☆11May 14, 2022Updated 4 years ago
- Repository to host micro service implementation patterns.☆14Jun 25, 2025Updated last year
- This is an end to end MLOps system☆34Nov 27, 2025Updated 7 months ago
- Get data from API, run a scheduled script with Airflow, send data to Kafka and consume with Spark, then write to Cassandra☆146Jul 27, 2023Updated 2 years ago
- Integrating with Spotify API and extracting Data. Deploying code on AWS Lambda for Data Extraction. Adding trigger to run the extraction …☆12Jul 5, 2023Updated 2 years ago
- ☆30Nov 16, 2023Updated 2 years ago
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- GPT-4o Powered Calorie Detecor☆18May 29, 2024Updated 2 years ago
- This solution helps you deploy ETL processes and data storage resources to create an Insurance Lake using Amazon S3 buckets for storage, …☆36Mar 12, 2026Updated 3 months ago
- text2sql with modern LLMs (duckdb-nsql, SQLCoder etc ...)☆18Apr 13, 2024Updated 2 years ago
- Produce Kafka messages, consume them and upload into Cassandra, MongoDB.☆43Sep 26, 2023Updated 2 years ago
- Use a AWS Glue Python Shell Job to connect to your Amazon Redshift cluster and execute a SQL script stored in Amazon S3.☆21Aug 8, 2022Updated 3 years ago
- A demonstration of an ELT (Extract, Load, Transform) pipeline☆32Feb 19, 2024Updated 2 years ago
- Spark application to consume kafka events generated by a python producer.☆12Aug 7, 2021Updated 4 years ago
- A platform that helps developers to better understand CSS through declaration interpretation and may even improve them through suggestion…☆14Jul 3, 2021Updated 4 years ago
- This repository has moved into https://github.com/dbt-labs/dbt-adapters☆53Feb 7, 2025Updated last year
- Managed Database hosting by DigitalOcean • AdPostgreSQL, MySQL, MongoDB, Kafka, Valkey, and OpenSearch available. Automatically scale up storage and focus on building your apps.
- TTS utility☆12Aug 2, 2020Updated 5 years ago
- Spark Notebook docker image☆10Dec 29, 2017Updated 8 years ago
- THREE JS 2D buttons and labels achorable library☆13Nov 29, 2016Updated 9 years ago
- This formatter which is for handling parameters and file uploaded to Web API controller.☆26Dec 7, 2022Updated 3 years ago
- The proposed solution shows and approach to unify and centralize logs across different compute platforms like EC2, ECS, EKS and Lambda wi…☆14Oct 17, 2023Updated 2 years ago
- ☆15Nov 16, 2024Updated last year
- ☆13Jan 6, 2022Updated 4 years ago