This project introduces PySpark, a powerful open-source framework for distributed data processing. We explore its architecture, components, and applications for real-time data analysis.
☆47Sep 26, 2024Updated 2 years ago
Alternatives and similar repositories for Real-Time-PySpark
Users that are interested in Real-Time-PySpark are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- A Python PySpark Projet with Poetry☆32May 2, 2026Updated 6 months ago
- This repo consists of all important concepts for data engineers.☆11Jun 2, 2026Updated 5 months ago
- ☆96Sep 14, 2022Updated 4 years ago
- This repository contains code and configuration files for an Extract, Transform, Load (ETL) project using Google Cloud Data Fusion for da…☆21Feb 23, 2024Updated 2 years ago
- Build and run Spark Structured Streaming pipelines in Hadoop - project using PySpark.☆13Jun 6, 2019Updated 7 years ago
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- Apache Airflow advanced functionalities examples☆21Mar 22, 2024Updated 2 years ago
- ☆24Sep 21, 2020Updated 6 years ago
- Implementing best practices for PySpark ETL jobs and applications.☆2,121Jan 1, 2023Updated 3 years ago
- Examples of Apache Flink® v2.1 applications showcasing the DataStream API, Table API in Java and Python, and Flink SQL, featuring AWS, Gi…☆10Jan 13, 2026Updated 10 months ago
- ☆18Jul 26, 2025Updated last year
- pyspark dataframe made easy☆16Dec 15, 2021Updated 4 years ago
- End-to-end data pipeline that ingests, processes, and stores data. It uses Apache Airflow to schedule scripts that fetch data from an API…☆23Sep 23, 2026Updated last month
- ☆39Oct 27, 2024Updated 2 years ago
- ☆14Mar 9, 2025Updated last year
- AI Agents on DigitalOcean Gradient AI Platform • AdBuild production-ready AI agents using customizable tools or access multiple LLMs through a single endpoint. Create custom knowledge bases or connect external data.
- This is the first project where we worked on apache spark, In this project what we have done is that we downloaded the datasets from KAGG…☆23Oct 14, 2021Updated 5 years ago
- Demonstrating the efficiency of pmdarima’s auto_arima() function compared to implementing a traditional ARIMA model.☆12Feb 16, 2021Updated 5 years ago
- ☆20Mar 28, 2022Updated 4 years ago
- This repo contains notes on SQL and PostgreSQL☆46Oct 6, 2022Updated 4 years ago
- ☆12Aug 16, 2020Updated 6 years ago
- This repo contains all the code used in the Python for Data Engineering Course☆388Apr 24, 2024Updated 2 years ago
- Automated and tool agnostic data integration testing tool.☆11Mar 29, 2022Updated 4 years ago
- Ease With Data☆11Mar 19, 2023Updated 3 years ago
- ☆29Dec 13, 2021Updated 4 years ago
- Proton VPN Special Offer - Get 70% off • AdSpecial partner offer. Trusted by over 100 million users worldwide. Tested, Approved and Recommended by Experts.
- About Let The Data Confess☆10May 2, 2021Updated 5 years ago
- Building an PD, LGD and EAD Model for Financial Modeling.☆15Dec 19, 2023Updated 2 years ago
- Learn PySpark from Basics to Advanced. Checkout the YouTube Series : [PySpark - Zero to Hero]☆171Sep 7, 2025Updated last year
- Datasets used in PyCaret☆12Mar 3, 2022Updated 4 years ago
- BES (Bigdata Ecosystem Sandbox) is an all-in-one development and learning environment designed to simplify the exploration, experimentati…☆11Nov 18, 2024Updated 2 years ago
- 😎 Awesome list of Retrieval-Augmented Generation (RAG) applications in Generative AI.☆13May 30, 2026Updated 5 months ago
- An AWS Data Engineering End-to-End Project (Glue, Lambda, Kinesis, Redshift, QuickSight, Athena, EC2, S3)☆17Sep 20, 2023Updated 3 years ago
- Fundamentals of Spark with Python (using PySpark), code examples☆363Oct 29, 2022Updated 4 years ago
- Scikit-Learn Tutorials with 200 videos and lectures☆12Aug 6, 2024Updated 2 years ago
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- Projects done in the Data Engineer Nanodegree Program by Udacity.com☆182Dec 8, 2022Updated 3 years ago
- This library has moved to https://github.com/googleapis/google-cloud-python/tree/main/packages/google-cloud-error-reporting☆28Feb 23, 2026Updated 8 months ago
- Lists of publicly available datasets for machine learning☆16Mar 23, 2024Updated 2 years ago
- A dataset listing Government of Canada IT projects over $1M☆11Apr 23, 2024Updated 2 years ago
- ☆12Feb 15, 2019Updated 7 years ago
- ☆11Jun 15, 2019Updated 7 years ago
- Big data projects☆12Dec 10, 2018Updated 7 years ago
Are these results useful?