An end-to-end data engineering pipeline that fetches data from Wikipedia, cleans and transforms it with Apache Airflow and saves it on Azure Data Lake. Other processing takes place on Azure Data Factory, Azure Synapse and Tableau.
☆32Oct 2, 2023Updated 2 years ago
Alternatives and similar repositories for FootballDataEngineering
Users that are interested in FootballDataEngineering are comparing it to the libraries listed below
Sorting:
- This project provides an end-to-end data processing and visualization of visa numbers in Japan using PySpark and Plotly. The spark cluste…☆12Oct 11, 2023Updated 2 years ago
- This project serves as a comprehensive guide to building an end-to-end data engineering pipeline using TCP/IP Socket, Apache Spark, OpenA…☆43Jan 4, 2024Updated 2 years ago
- This repository contains an end-to-end data engineering project using Apache Flink, focused on performing sales analytics. The project de…☆11Nov 18, 2023Updated 2 years ago
- ☆13Feb 14, 2025Updated last year
- An end-to-end data engineering pipeline that fetches real-time YouTube analytics and streams them through Kafka for processing with ksqlD…☆16Sep 19, 2023Updated 2 years ago
- Includes all the Practice Material and Project☆23May 19, 2025Updated 10 months ago
- This project shows how to capture changes from postgres database and stream them into kafka☆42May 17, 2024Updated last year
- This repository contains an Apache Flink application for real-time sales analytics built using Docker Compose to orchestrate the necessar…☆48Dec 4, 2023Updated 2 years ago
- This repository contains the code for a realtime election voting system. The system is built using Python, Kafka, Spark Streaming, Postgr…☆45Dec 11, 2023Updated 2 years ago
- This project demonstrates how to use Apache Airflow to submit jobs to Apache spark cluster in different programming laguages using Python…☆48Mar 14, 2024Updated 2 years ago
- Snowflake - Build and Architect Data Pipelines using AWS, published by Packt☆23Apr 3, 2023Updated 2 years ago
- ms-dataverse is a Python module for Microsoft Dataverse, offering a lightweight ORM to query, create, update, and delete entities. Utiliz…☆13Apr 10, 2023Updated 2 years ago
- This project provides a comprehensive data pipeline solution to extract, transform, and load (ETL) Reddit data into a Redshift data wareh…☆208Oct 23, 2023Updated 2 years ago
- The official documentation of the City of Boston's Analytics Team.☆13Jan 21, 2025Updated last year
- ☆17Feb 1, 2025Updated last year
- ☆12Aug 8, 2023Updated 2 years ago
- Data pipeline from device to cloud☆12May 14, 2022Updated 3 years ago
- This is a professional training designed by Google with 5 courses. This program also prepares one for the CompTIA A+ exams, the industry …☆10Nov 26, 2022Updated 3 years ago
- This is an end to end MLOps system☆34Nov 27, 2025Updated 3 months ago
- Prediction of traffic patterns in bike sharing systems. Including dashboard for clustering analysis of stations in bike share networks ba…☆11Sep 22, 2022Updated 3 years ago
- Transform data from on-premises SQL Server to Azure Delta Lake Storage for Analytics and Visualization☆22Jul 16, 2023Updated 2 years ago
- Toolset for detecting reflected xss in websites☆16Oct 6, 2018Updated 7 years ago
- ☆12Jan 14, 2023Updated 3 years ago
- ☆31Nov 14, 2024Updated last year
- ☆24Dec 31, 2024Updated last year
- Python wrapper for Goodreads API☆30Feb 20, 2020Updated 6 years ago
- ☆41Dec 5, 2023Updated 2 years ago
- Functional Data Engineering tutorial in Python & Airflow.☆17Mar 24, 2023Updated 2 years ago
- Local SQL Database ---> Azure ---> Power BI☆14Oct 13, 2023Updated 2 years ago
- ☆15Aug 5, 2023Updated 2 years ago
- With everything I learned from DEZoomcamp from datatalks.club, this project performs a batch processing on AWS for the cycling dataset wh…☆15Jan 4, 2026Updated 2 months ago
- End-to-end Data Project (DA/DS/DE/MLOps) - retail/e-commerce - interpretable dynamic clustering☆16Jul 12, 2025Updated 8 months ago
- ☆22May 13, 2019Updated 6 years ago
- Repository for Data Engineering Interview Series☆36Oct 17, 2024Updated last year
- Books and materials for learning Electrical, Electronics and Communication Engineering☆24May 2, 2020Updated 5 years ago
- Data Engineering Project: Extracting music video metrics of Twice using YouTube API, AWS, and Tableau☆32Nov 21, 2023Updated 2 years ago
- ☆15Aug 3, 2022Updated 3 years ago
- ELT Data Pipeline implementation in Data Warehousing environment☆30May 2, 2025Updated 10 months ago
- A course by DataTalks Club that covers Spark, Kafka, Docker, Airflow, Terraform, DBT, Big Query etc☆16Mar 18, 2022Updated 4 years ago