NitinDatta8 / realtime-data-streamingLinks
End-to-end data engineering pipeline with various technologies to ingest real time data.
☆17Updated last year
Alternatives and similar repositories for realtime-data-streaming
Users that are interested in realtime-data-streaming are comparing it to the libraries listed below
Sorting:
- An end-to-end data engineering pipeline that orchestrates data ingestion, processing, and storage using Apache Airflow, Python, Apache Ka…☆282Updated 8 months ago
- Hands-on MLOps projects to explore and learn the practical aspects of machine learning engineering for production.☆80Updated 6 months ago
- Data Engineering portfolio projects, resources used to study data tools...☆27Updated last year
- Cool DE Projects☆36Updated 2 months ago
- Learn PySpark from Basics to Advanced. Checkout the YouTube Series : [PySpark - Zero to Hero]☆90Updated last month
- End-to-end Data Project (DA/DS/DE/MLOps) - retail/e-commerce - interpretable dynamic clustering☆15Updated 3 months ago
- Nyc_Taxi_Data_Pipeline - DE Project☆126Updated 11 months ago
- This project provides a comprehensive data pipeline solution to extract, transform, and load (ETL) Reddit data into a Redshift data wareh…☆158Updated last year
- ☆49Updated 11 months ago
- Production ML rental prediction system.☆48Updated last year
- End to end data engineering project with kafka, airflow, spark, postgres and docker.☆103Updated 6 months ago
- ☆292Updated last year
- ☆106Updated 2 years ago
- A data engineering project with Kafka, Spark Streaming, dbt, Docker, Airflow, Terraform, GCP and much more!☆753Updated 3 years ago
- This is a repository to demonstrate my details, skills, projects and to keep track of my progression in Data Analytics and Data Science t…☆135Updated 9 months ago
- ☆12Updated 11 months ago
- This repository is to show my Data Analytics & Engineering skills, share projects, and track my progress.☆57Updated 2 years ago
- This project serves as a comprehensive guide to building an end-to-end data engineering pipeline using TCP/IP Socket, Apache Spark, OpenA…☆44Updated last year
- This project introduces PySpark, a powerful open-source framework for distributed data processing. We explore its architecture, component…☆34Updated last year
- Personal Data Engineering Projects☆955Updated 2 years ago
- Stream processing pipeline from Finnhub websocket using Spark, Kafka, Kubernetes and more☆365Updated last year
- ☆357Updated last year
- PySpark Tutorial for Beginners - Practical Examples in Jupyter Notebook with Spark version 3.4.1. The tutorial covers various topics like…☆134Updated 2 years ago
- ☆16Updated 3 years ago
- Welcome to my data engineering projects repository! Here you will find a collection of data engineering projects that I have worked on.☆22Updated 2 years ago
- Roadmap for Data Engineering☆237Updated last year
- My Insight Data Engineering Fellowship project. I implemented a big data processing pipeline based on lambda architecture, that aggrega…☆505Updated 3 years ago
- This repo contains all the code used in the Python for Data Engineering Course☆321Updated last year
- Machine Learning In Production (MLOps)☆206Updated last week
- This repo demonstrates the development of a real-time data pipeline designed to ingest, process, and analyze stock market data. Using cut…☆45Updated last year