An end-to-end data engineering pipeline that fetches data from Wikipedia, cleans and transforms it with Apache Airflow and saves it on Azure Data Lake. Other processing takes place on Azure Data Factory, Azure Synapse and Tableau.
☆30Oct 2, 2023Updated 2 years ago
Alternatives and similar repositories for FootballDataEngineering
Users that are interested in FootballDataEngineering are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- This project provides an end-to-end data processing and visualization of visa numbers in Japan using PySpark and Plotly. The spark cluste…☆11Oct 11, 2023Updated 2 years ago
- This project serves as a comprehensive guide to building an end-to-end data engineering pipeline using TCP/IP Socket, Apache Spark, OpenA…☆46Jan 4, 2024Updated 2 years ago
- This project showcases how to integrate the world of DevOps, focusing on Continuous Integration (CI) and Continuous Deployment (CD) with …☆14Dec 27, 2023Updated 2 years ago
- An end-to-end data engineering pipeline that fetches real-time YouTube analytics and streams them through Kafka for processing with ksqlD…☆16Sep 19, 2023Updated 3 years ago
- This project provides a comprehensive data pipeline solution to extract, transform, and load (ETL) Reddit data into a Redshift data wareh…☆228Oct 23, 2023Updated 2 years ago
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- An end-to-end data engineering pipeline that orchestrates data ingestion, processing, and storage using Apache Airflow, Python, Apache Ka…☆360Feb 14, 2025Updated last year
- Transform data from on-premises SQL Server to Azure Delta Lake Storage for Analytics and Visualization☆29Jul 16, 2023Updated 3 years ago
- This demo shows how to stream data to cloud databases with Confluent. It includes fully-managed connectors (Oracle CDC, RabbitMQ, MongoDB…☆11Jan 10, 2025Updated last year
- This project shows how to capture changes from postgres database and stream them into kafka☆43May 17, 2024Updated 2 years ago
- This repository contains an Apache Flink application for real-time sales analytics built using Docker Compose to orchestrate the necessar…☆52Dec 4, 2023Updated 2 years ago
- ☆47Jul 6, 2024Updated 2 years ago
- This repository contains the code for a realtime election voting system. The system is built using Python, Kafka, Spark Streaming, Postgr…☆50Dec 11, 2023Updated 2 years ago
- This project demonstrates how to use Apache Airflow to submit jobs to Apache spark cluster in different programming laguages using Python…☆48Mar 14, 2024Updated 2 years ago
- Quantum Black Hackathon organised by Analytics Vidya☆13Jul 23, 2019Updated 7 years ago
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- ☆17Aug 2, 2025Updated last year
- The official documentation of the City of Boston's Analytics Team.☆13Jan 21, 2025Updated last year
- Create a streaming data, transfer it to Kafka, modify it with PySpark, take it to ElasticSearch and MinIO☆66Jul 21, 2023Updated 3 years ago
- End-to-end ETL pipeline in the Microsoft Azure cloud - (Jun '24 - Jul '24)☆41Aug 9, 2024Updated 2 years ago
- ☆17Mar 10, 2025Updated last year
- Resources for tracking DP-700 exam prep progress.☆58Jun 14, 2025Updated last year
- ☆18Feb 1, 2025Updated last year
- Toolset for detecting reflected xss in websites☆16Oct 6, 2018Updated 7 years ago
- Local SQL Database ---> Azure ---> Power BI☆15Oct 13, 2023Updated 2 years ago
- Wordpress hosting with auto-scaling - Free Trial Offer • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- Scrapper and analyzer of shared scooter data☆10Jul 30, 2024Updated 2 years ago
- ☆12Jan 14, 2023Updated 3 years ago
- ☆29Apr 16, 2026Updated 5 months ago
- Welcome to a fantastic place to start learning to code! This repo is dedicated to making the learning process easy, fun, and enjoyable!☆16Jan 26, 2021Updated 5 years ago
- ☆28Dec 31, 2024Updated last year
- Data Structures and Algorithms☆22May 24, 2026Updated 3 months ago
- This repository showcases a collection of machine learning projects in various domains, demonstrating my skills and expertise as a data s…☆12Nov 20, 2023Updated 2 years ago
- ☆16Aug 5, 2023Updated 3 years ago
- Functional Data Engineering tutorial in Python & Airflow.☆17Mar 24, 2023Updated 3 years ago
- Wordpress hosting with auto-scaling - Free Trial Offer • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- End-to-end Data Project (DA/DS/DE/MLOps) - retail/e-commerce - interpretable dynamic clustering☆23Jul 12, 2025Updated last year
- With everything I learned from DEZoomcamp from datatalks.club, this project performs a batch processing on AWS for the cycling dataset wh…☆15Jan 4, 2026Updated 8 months ago
- Dockerized analytics pipeline where Airflow moves mock-API data into Postgres, runs dbt transformations, and serves clean tables for Powe…☆84Oct 18, 2025Updated 11 months ago
- ☆33Jul 29, 2023Updated 3 years ago
- This project demonstrates real-time data streaming and processing architecture using Kafka, Spark Streaming, and Debezium for capturing C…☆19Oct 24, 2024Updated last year
- ☆14Oct 1, 2022Updated 3 years ago
- ☆23May 13, 2019Updated 7 years ago