This project introduces PySpark, a powerful open-source framework for distributed data processing. We explore its architecture, components, and applications for real-time data analysis.
☆47Sep 26, 2024Updated 2 years ago
Alternatives and similar repositories for Real-Time-PySpark
Users that are interested in Real-Time-PySpark are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- A Python PySpark Projet with Poetry☆31May 2, 2026Updated 5 months ago
- This repo consists of all important concepts for data engineers.☆11Jun 2, 2026Updated 4 months ago
- ☆95Sep 14, 2022Updated 4 years ago
- ☆18Jul 26, 2025Updated last year
- Build and run Spark Structured Streaming pipelines in Hadoop - project using PySpark.☆13Jun 6, 2019Updated 7 years ago
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- Apache Airflow advanced functionalities examples☆21Mar 22, 2024Updated 2 years ago
- End-to-end data pipeline that ingests, processes, and stores data. It uses Apache Airflow to schedule scripts that fetch data from an API…☆23Sep 23, 2026Updated last month
- My solutions for the Udacity Data Engineering Nanodegree☆33Oct 14, 2019Updated 7 years ago
- Examples of Apache Flink® v2.1 applications showcasing the DataStream API, Table API in Java and Python, and Flink SQL, featuring AWS, Gi…☆10Jan 13, 2026Updated 9 months ago
- pyspark dataframe made easy☆16Dec 15, 2021Updated 4 years ago
- ☆20Mar 28, 2022Updated 4 years ago
- 😎 Awesome list of Retrieval-Augmented Generation (RAG) applications in Generative AI.☆13May 30, 2026Updated 5 months ago
- Tokens → Transformers → Model Selection → Providers → Prompt Engineering → Structured Outputs → Embeddings → Vector Search → RAG → Advanc…☆18Sep 9, 2026Updated last month
- ☆17May 26, 2023Updated 3 years ago
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- ☆29Dec 13, 2021Updated 4 years ago
- Simple project using pyflink, kafka and postgre containerized using Docker☆12Aug 26, 2024Updated 2 years ago
- BES (Bigdata Ecosystem Sandbox) is an all-in-one development and learning environment designed to simplify the exploration, experimentati…☆11Nov 18, 2024Updated last year
- ☆19Nov 22, 2025Updated 11 months ago
- Learn PySpark from Basics to Advanced. Checkout the YouTube Series : [PySpark - Zero to Hero]☆171Sep 7, 2025Updated last year
- Datasets used in PyCaret☆12Mar 3, 2022Updated 4 years ago
- ☆13Apr 14, 2024Updated 2 years ago
- Birgitta is a Python ETL test and schema framework, providing automated tests for pyspark notebooks/recipes.☆16Nov 9, 2023Updated 2 years ago
- Lists of publicly available datasets for machine learning☆16Mar 23, 2024Updated 2 years ago
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- An AWS Data Engineering End-to-End Project (Glue, Lambda, Kinesis, Redshift, QuickSight, Athena, EC2, S3)☆17Sep 20, 2023Updated 3 years ago
- This project serves as a comprehensive guide to building an end-to-end data engineering pipeline using TCP/IP Socket, Apache Spark, OpenA…☆47Jan 4, 2024Updated 2 years ago
- Projects done in the Data Engineer Nanodegree Program by Udacity.com☆180Dec 8, 2022Updated 3 years ago
- Nyc_Taxi_Data_Pipeline - DE Project☆156Oct 21, 2024Updated 2 years ago
- Big data projects☆12Dec 10, 2018Updated 7 years ago
- Demo code on agglomerative clustering☆12Nov 7, 2021Updated 4 years ago
- Config files for my GitHub profile.☆65Aug 21, 2026Updated 2 months ago
- ☆38Jun 3, 2026Updated 4 months ago
- Data Engineering YouTube Analysis Project by Darshil Parmar☆256Dec 8, 2023Updated 2 years ago
- Managed Kubernetes at scale on DigitalOcean • AdDigitalOcean Kubernetes includes the control plane, bandwidth allowance, container registry, automatic updates, and more for free.
- ☆11Jun 15, 2019Updated 7 years ago
- A repository for Analysis of Toronto Neighbourhoods (An IBM Data Science Capstone Project)☆10Jan 15, 2021Updated 5 years ago
- ☆12Feb 15, 2019Updated 7 years ago
- ☆20Dec 19, 2025Updated 10 months ago
- Deep Learning for NLP☆12Dec 7, 2022Updated 3 years ago
- ☆19Jun 2, 2025Updated last year
- Taipy Demo of a Realtime Dashboard of Air Pollution around a Factory☆18May 20, 2025Updated last year