An end-to-end data engineering pipeline that fetches data from Wikipedia, cleans and transforms it with Apache Airflow and saves it on Azure Data Lake. Other processing takes place on Azure Data Factory, Azure Synapse and Tableau.
☆32Oct 2, 2023Updated 3 years ago
Alternatives and similar repositories for FootballDataEngineering
Users that are interested in FootballDataEngineering are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- This project provides an end-to-end data processing and visualization of visa numbers in Japan using PySpark and Plotly. The spark cluste…☆11Oct 11, 2023Updated 3 years ago
- This project serves as a comprehensive guide to building an end-to-end data engineering pipeline using TCP/IP Socket, Apache Spark, OpenA…☆47Jan 4, 2024Updated 2 years ago
- Few projects related to Data Engineering including Data Modeling, Infrastructure setup on cloud, Data Warehousing and Data Lake developme…☆12Feb 26, 2020Updated 6 years ago
- This project showcases how to integrate the world of DevOps, focusing on Continuous Integration (CI) and Continuous Deployment (CD) with …☆14Dec 27, 2023Updated 2 years ago
- This repository contains an end-to-end data engineering project using Apache Flink, focused on performing sales analytics. The project de…☆12Nov 18, 2023Updated 3 years ago
- End-to-end encrypted cloud storage - Proton Drive • AdSpecial offer: 40% Off Yearly / 80% Off First Month. Protect your most important files, photos, and documents from prying eyes.
- An end-to-end data engineering pipeline that fetches real-time YouTube analytics and streams them through Kafka for processing with ksqlD…☆16Sep 19, 2023Updated 3 years ago
- In this project, we setup and end to end data engineering using Apache Spark, Azure Databricks, Data Build Tool (DBT) using Azure as our …☆40Dec 18, 2023Updated 2 years ago
- This project provides a comprehensive data pipeline solution to extract, transform, and load (ETL) Reddit data into a Redshift data wareh…☆230Oct 23, 2023Updated 3 years ago
- ☆27Dec 31, 2024Updated last year
- This repository contains an Apache Flink application for real-time sales analytics built using Docker Compose to orchestrate the necessar…☆53Dec 4, 2023Updated 2 years ago
- This repository contains the necessary configuration files and DAGs (Directed Acyclic Graphs) for setting up a robust data engineering en…☆25Jan 26, 2024Updated 2 years ago
- Airflow tutorial☆14Apr 2, 2022Updated 4 years ago
- Data Engineering Projects☆11Mar 30, 2024Updated 2 years ago
- Transform data from on-premises SQL Server to Azure Delta Lake Storage for Analytics and Visualization☆29Jul 16, 2023Updated 3 years ago
- Wordpress hosting with auto-scaling - Free Trial Offer • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- This demo shows how to stream data to cloud databases with Confluent. It includes fully-managed connectors (Oracle CDC, RabbitMQ, MongoDB…☆11Jan 10, 2025Updated last year
- This is the repository for "MCUCoder: Adaptive Bitrate Learned Video Compression for IoT Devices"☆13Feb 3, 2026Updated 9 months ago
- A data pipeline for processing football data using Python and SQL☆13Sep 12, 2023Updated 3 years ago
- ☆11Aug 8, 2023Updated 3 years ago
- Real-time streaming pipeline using Kafka, Spark Structured Streaming, and Delta Lake, processing millions of events per day with sub-minu…☆16Sep 30, 2025Updated last year
- ms-dataverse is a Python module for Microsoft Dataverse, offering a lightweight ORM to query, create, update, and delete entities. Utiliz…☆13Apr 10, 2023Updated 3 years ago
- This project shows how to capture changes from postgres database and stream them into kafka☆44May 17, 2024Updated 2 years ago
- Python or SQL for data transformation☆11Jul 4, 2024Updated 2 years ago
- The official documentation of the City of Boston's Analytics Team.☆13Jan 21, 2025Updated last year
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- A repository of solved projects in Data Engineering for beginners and professionals.☆16May 13, 2025Updated last year
- ☆17Aug 2, 2025Updated last year
- A Python wrapper for the Iterable API☆11Jan 7, 2026Updated 10 months ago
- This project demonstrates how to use Apache Airflow to submit jobs to Apache spark cluster in different programming laguages using Python…☆47Mar 14, 2024Updated 2 years ago
- Building Realtime End to End Sales Forecasting ML Pipeline☆15Jul 30, 2025Updated last year
- Create a streaming data, transfer it to Kafka, modify it with PySpark, take it to ElasticSearch and MinIO☆66Jul 21, 2023Updated 3 years ago
- Snowflake - Build and Architect Data Pipelines using AWS, published by Packt☆24Apr 3, 2023Updated 3 years ago
- End-to-end ETL pipeline in the Microsoft Azure cloud - (Jun '24 - Jul '24)☆41Aug 9, 2024Updated 2 years ago
- Material for the course "Introduction to Apache Spark APIs for Data Processing" https://sparktraining.web.cern.ch/☆21May 13, 2025Updated last year
- Wordpress hosting with auto-scaling - Free Trial Offer • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- A real-time financial data streaming pipeline and visualization platform using Apache Kafka, Cassandra, and Bokeh.☆16Nov 6, 2026Updated 2 weeks ago
- ☆17Mar 10, 2025Updated last year
- Toolset for detecting reflected xss in websites☆16Oct 6, 2018Updated 8 years ago
- Local SQL Database ---> Azure ---> Power BI☆15Oct 13, 2023Updated 3 years ago
- Creates csv files containing football data scraped from the website www.fbref.com☆26Jul 4, 2023Updated 3 years ago
- ☆12Jan 14, 2023Updated 3 years ago
- ☆20Feb 14, 2025Updated last year
Are these results useful?