An end-to-end data engineering pipeline that fetches data from Wikipedia, cleans and transforms it with Apache Airflow and saves it on Azure Data Lake. Other processing takes place on Azure Data Factory, Azure Synapse and Tableau.
☆32Oct 2, 2023Updated 2 years ago
Alternatives and similar repositories for FootballDataEngineering
Users that are interested in FootballDataEngineering are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- This project serves as a comprehensive guide to building an end-to-end data engineering pipeline using TCP/IP Socket, Apache Spark, OpenA…☆44Jan 4, 2024Updated 2 years ago
- This project showcases how to integrate the world of DevOps, focusing on Continuous Integration (CI) and Continuous Deployment (CD) with …☆14Dec 27, 2023Updated 2 years ago
- This repository contains an end-to-end data engineering project using Apache Flink, focused on performing sales analytics. The project de…☆11Nov 18, 2023Updated 2 years ago
- In this project, we setup and end to end data engineering using Apache Spark, Azure Databricks, Data Build Tool (DBT) using Azure as our …☆39Dec 18, 2023Updated 2 years ago
- An end-to-end data engineering pipeline that fetches real-time YouTube analytics and streams them through Kafka for processing with ksqlD…☆16Sep 19, 2023Updated 2 years ago
- DigitalOcean Gradient AI Platform • AdBuild production-ready AI agents using customizable tools or access multiple LLMs through a single endpoint. Create custom knowledge bases or connect external data.
- This repository contains the necessary configuration files and DAGs (Directed Acyclic Graphs) for setting up a robust data engineering en…☆25Jan 26, 2024Updated 2 years ago
- An end-to-end data engineering pipeline that orchestrates data ingestion, processing, and storage using Apache Airflow, Python, Apache Ka…☆323Feb 14, 2025Updated last year
- This project shows how to capture changes from postgres database and stream them into kafka☆42May 17, 2024Updated last year
- A data pipeline for processing football data using Python and SQL☆13Sep 12, 2023Updated 2 years ago
- This repository contains an Apache Flink application for real-time sales analytics built using Docker Compose to orchestrate the necessar…☆50Dec 4, 2023Updated 2 years ago
- This repository contains the code for a realtime election voting system. The system is built using Python, Kafka, Spark Streaming, Postgr…☆47Dec 11, 2023Updated 2 years ago
- ☆41Jul 6, 2024Updated last year
- A Python wrapper for the Iterable API☆12Jan 7, 2026Updated 3 months ago
- Snowflake - Build and Architect Data Pipelines using AWS, published by Packt☆23Apr 3, 2023Updated 3 years ago
- 1-Click AI Models by DigitalOcean Gradient • AdDeploy popular AI models on DigitalOcean Gradient GPU virtual machines with just a single click and start building anything your business needs.
- This project provides a comprehensive data pipeline solution to extract, transform, and load (ETL) Reddit data into a Redshift data wareh…☆212Oct 23, 2023Updated 2 years ago
- Create a streaming data, transfer it to Kafka, modify it with PySpark, take it to ElasticSearch and MinIO☆65Jul 21, 2023Updated 2 years ago
- ☆17Feb 1, 2025Updated last year
- ☆17Mar 10, 2025Updated last year
- Data pipeline from device to cloud☆12May 14, 2022Updated 3 years ago
- Prediction of traffic patterns in bike sharing systems. Including dashboard for clustering analysis of stations in bike share networks ba…☆12Sep 22, 2022Updated 3 years ago
- Making and developing R packages☆20Mar 25, 2026Updated 2 weeks ago
- Toolset for detecting reflected xss in websites☆16Oct 6, 2018Updated 7 years ago
- Transform data from on-premises SQL Server to Azure Delta Lake Storage for Analytics and Visualization☆22Jul 16, 2023Updated 2 years ago
- DigitalOcean Gradient AI Platform • AdBuild production-ready AI agents using customizable tools or access multiple LLMs through a single endpoint. Create custom knowledge bases or connect external data.
- This project demonstrates real-time data streaming and processing architecture using Kafka, Spark Streaming, and Debezium for capturing C…☆13Oct 24, 2024Updated last year
- A complete computer science study plan to become a software engineer.☆11Feb 17, 2024Updated 2 years ago
- Scrapper and analyzer of shared scooter data☆11Jul 30, 2024Updated last year
- ☆12Jan 14, 2023Updated 3 years ago
- ☆31Nov 14, 2024Updated last year
- Data Structures and Algorithms☆21Feb 23, 2026Updated last month
- Python wrapper for Goodreads API☆30Feb 20, 2020Updated 6 years ago
- ☆28Jul 9, 2025Updated 9 months ago
- Functional Data Engineering tutorial in Python & Airflow.☆17Mar 24, 2023Updated 3 years ago
- Wordpress hosting with auto-scaling on Cloudways • AdFully Managed hosting built for WordPress-powered businesses that need reliable, auto-scalable hosting. Cloudways SafeUpdates now available.
- Project exploring data collection, visualisation and analysis of Sports Statistics.☆14Dec 17, 2020Updated 5 years ago
- ☆30Jul 29, 2023Updated 2 years ago
- End-to-end Data Project (DA/DS/DE/MLOps) - retail/e-commerce - interpretable dynamic clustering☆18Jul 12, 2025Updated 8 months ago
- This formatter which is for handling parameters and file uploaded to Web API controller.☆26Dec 7, 2022Updated 3 years ago
- ☆22May 13, 2019Updated 6 years ago
- This project leverages GCS, Composer, Dataflow, BigQuery, and Looker on Google Cloud Platform (GCP) to build a robust data engineering so…☆35Dec 12, 2023Updated 2 years ago
- Data Engineering Project: Extracting music video metrics of Twice using YouTube API, AWS, and Tableau☆32Nov 21, 2023Updated 2 years ago