An end-to-end data engineering pipeline that fetches data from Wikipedia, cleans and transforms it with Apache Airflow and saves it on Azure Data Lake. Other processing takes place on Azure Data Factory, Azure Synapse and Tableau.
☆31Oct 2, 2023Updated 3 years ago
Alternatives and similar repositories for FootballDataEngineering
Users that are interested in FootballDataEngineering are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- This project provides an end-to-end data processing and visualization of visa numbers in Japan using PySpark and Plotly. The spark cluste…☆11Oct 11, 2023Updated 3 years ago
- This project serves as a comprehensive guide to building an end-to-end data engineering pipeline using TCP/IP Socket, Apache Spark, OpenA…☆47Jan 4, 2024Updated 2 years ago
- Few projects related to Data Engineering including Data Modeling, Infrastructure setup on cloud, Data Warehousing and Data Lake developme…☆12Feb 26, 2020Updated 6 years ago
- This repository contains an end-to-end data engineering project using Apache Flink, focused on performing sales analytics. The project de…☆12Nov 18, 2023Updated 2 years ago
- In this project, we setup and end to end data engineering using Apache Spark, Azure Databricks, Data Build Tool (DBT) using Azure as our …☆40Dec 18, 2023Updated 2 years ago
- Managed Kubernetes at scale on DigitalOcean • AdDigitalOcean Kubernetes includes the control plane, bandwidth allowance, container registry, automatic updates, and more for free.
- An end-to-end data engineering pipeline that fetches real-time YouTube analytics and streams them through Kafka for processing with ksqlD…☆16Sep 19, 2023Updated 3 years ago
- This project provides a comprehensive data pipeline solution to extract, transform, and load (ETL) Reddit data into a Redshift data wareh…☆229Oct 23, 2023Updated 3 years ago
- ☆28Dec 31, 2024Updated last year
- This repository contains the code for a realtime election voting system. The system is built using Python, Kafka, Spark Streaming, Postgr…☆50Dec 11, 2023Updated 2 years ago
- A data pipeline for processing football data using Python and SQL☆13Sep 12, 2023Updated 3 years ago
- This is the repository for "MCUCoder: Adaptive Bitrate Learned Video Compression for IoT Devices"☆12Feb 3, 2026Updated 8 months ago
- Transform data from on-premises SQL Server to Azure Delta Lake Storage for Analytics and Visualization☆29Jul 16, 2023Updated 3 years ago
- This demo shows how to stream data to cloud databases with Confluent. It includes fully-managed connectors (Oracle CDC, RabbitMQ, MongoDB…☆11Jan 10, 2025Updated last year
- dbt Cloud plugin for Airflow☆13Apr 21, 2025Updated last year
- GPU virtual machines on DigitalOcean Gradient AI • AdGet to production fast with high-performance AMD and NVIDIA GPUs you can spin up in seconds. The definition of operational simplicity.
- ☆11Aug 8, 2023Updated 3 years ago
- ☆47Jul 6, 2024Updated 2 years ago
- Python or SQL for data transformation☆11Jul 4, 2024Updated 2 years ago
- Snowflake - Build and Architect Data Pipelines using AWS, published by Packt☆24Apr 3, 2023Updated 3 years ago
- This project demonstrates how to use Apache Airflow to submit jobs to Apache spark cluster in different programming laguages using Python…☆47Mar 14, 2024Updated 2 years ago
- Create a streaming data, transfer it to Kafka, modify it with PySpark, take it to ElasticSearch and MinIO☆66Jul 21, 2023Updated 3 years ago
- A real-time financial data streaming pipeline and visualization platform using Apache Kafka, Cassandra, and Bokeh.☆16Oct 23, 2026Updated last week
- Example orchestration pipeline for Fivetran + dbt managed by Airflow☆22Feb 18, 2021Updated 5 years ago
- Prediction of traffic patterns in bike sharing systems. Including dashboard for clustering analysis of stations in bike share networks ba…☆12Sep 22, 2022Updated 4 years ago
- Wordpress hosting with auto-scaling - Free Trial Offer • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- End-to-end ETL pipeline in the Microsoft Azure cloud - (Jun '24 - Jul '24)☆41Aug 9, 2024Updated 2 years ago
- ☆11Jul 21, 2025Updated last year
- Scrapper and analyzer of shared scooter data☆10Jul 30, 2024Updated 2 years ago
- ☆18Feb 1, 2025Updated last year
- ☆17Mar 10, 2025Updated last year
- ☆20Feb 14, 2025Updated last year
- Creates csv files containing football data scraped from the website www.fbref.com☆26Jul 4, 2023Updated 3 years ago
- Local SQL Database ---> Azure ---> Power BI☆15Oct 13, 2023Updated 3 years ago
- Resources for tracking DP-700 exam prep progress.☆61Jun 14, 2025Updated last year
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- ☆13Jun 14, 2024Updated 2 years ago
- ☆12Jan 14, 2023Updated 3 years ago
- ☆28Jun 28, 2024Updated 2 years ago
- Build a secure, production-ready MCP server for any project — connect Claude, ChatGPT & any AI agent to your backend with a URL + API key…☆17Sep 11, 2026Updated last month
- AI Reliability Engineering - ARIA☆14Aug 5, 2026Updated 2 months ago
- Self-hosted multi-user Apache Spark notebooks — PySpark + Scala, per-user kernel pods, MinIO IAM storage, OAuth. One Helm install. Apache…☆30Oct 14, 2026Updated 2 weeks ago
- Python wrapper for Goodreads API☆30Feb 20, 2020Updated 6 years ago