An end-to-end data engineering pipeline that fetches data from Wikipedia, cleans and transforms it with Apache Airflow and saves it on Azure Data Lake. Other processing takes place on Azure Data Factory, Azure Synapse and Tableau.
☆30Oct 2, 2023Updated 2 years ago
Alternatives and similar repositories for FootballDataEngineering
Users that are interested in FootballDataEngineering are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- This project provides an end-to-end data processing and visualization of visa numbers in Japan using PySpark and Plotly. The spark cluste…☆11Oct 11, 2023Updated 2 years ago
- This project serves as a comprehensive guide to building an end-to-end data engineering pipeline using TCP/IP Socket, Apache Spark, OpenA…☆45Jan 4, 2024Updated 2 years ago
- This project showcases how to integrate the world of DevOps, focusing on Continuous Integration (CI) and Continuous Deployment (CD) with …☆14Dec 27, 2023Updated 2 years ago
- This repository contains an end-to-end data engineering project using Apache Flink, focused on performing sales analytics. The project de…☆12Nov 18, 2023Updated 2 years ago
- In this project, we setup and end to end data engineering using Apache Spark, Azure Databricks, Data Build Tool (DBT) using Azure as our …☆38Dec 18, 2023Updated 2 years ago
- Virtual machines for every use case on DigitalOcean • AdGet dependable uptime with 99.99% SLA, simple security tools, and predictable monthly pricing with DigitalOcean's virtual machines, called Droplets.
- An end-to-end data engineering pipeline that fetches real-time YouTube analytics and streams them through Kafka for processing with ksqlD…☆16Sep 19, 2023Updated 2 years ago
- This repository contains the necessary configuration files and DAGs (Directed Acyclic Graphs) for setting up a robust data engineering en…☆25Jan 26, 2024Updated 2 years ago
- Transform data from on-premises SQL Server to Azure Delta Lake Storage for Analytics and Visualization☆27Jul 16, 2023Updated 3 years ago
- This demo shows how to stream data to cloud databases with Confluent. It includes fully-managed connectors (Oracle CDC, RabbitMQ, MongoDB…☆11Jan 10, 2025Updated last year
- Produce Kafka messages, consume them and upload into Cassandra, MongoDB.☆43Sep 26, 2023Updated 2 years ago
- This repository contains an Apache Flink application for real-time sales analytics built using Docker Compose to orchestrate the necessar…☆51Dec 4, 2023Updated 2 years ago
- ☆46Jul 6, 2024Updated 2 years ago
- A Python wrapper for the Iterable API☆12Jan 7, 2026Updated 6 months ago
- Helpful Resources for COGS 108 Students☆10Apr 6, 2026Updated 3 months ago
- GPU virtual machines on DigitalOcean Gradient AI • AdGet to production fast with high-performance AMD and NVIDIA GPUs you can spin up in seconds. The definition of operational simplicity.
- ☆12Aug 8, 2023Updated 2 years ago
- ☆16Oct 17, 2024Updated last year
- This project demonstrates how to use Apache Airflow to submit jobs to Apache spark cluster in different programming laguages using Python…☆48Mar 14, 2024Updated 2 years ago
- ms-dataverse is a Python module for Microsoft Dataverse, offering a lightweight ORM to query, create, update, and delete entities. Utiliz…☆13Apr 10, 2023Updated 3 years ago
- Prediction of traffic patterns in bike sharing systems. Including dashboard for clustering analysis of stations in bike share networks ba…☆12Sep 22, 2022Updated 3 years ago
- ☆17Aug 2, 2025Updated 11 months ago
- This project provides a comprehensive data pipeline solution to extract, transform, and load (ETL) Reddit data into a Redshift data wareh…☆220Oct 23, 2023Updated 2 years ago
- Snowflake - Build and Architect Data Pipelines using AWS, published by Packt☆24Apr 3, 2023Updated 3 years ago
- Creates csv files containing football data scraped from the website www.fbref.com☆24Jul 4, 2023Updated 3 years ago
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- End-to-end ETL pipeline in the Microsoft Azure cloud - (Jun '24 - Jul '24)☆40Aug 9, 2024Updated last year
- ☆18Feb 1, 2025Updated last year
- Data pipeline from device to cloud☆11May 14, 2022Updated 4 years ago
- ☆17Feb 14, 2025Updated last year
- Toolset for detecting reflected xss in websites☆16Oct 6, 2018Updated 7 years ago
- Scrapper and analyzer of shared scooter data☆11Jul 30, 2024Updated last year
- A complete computer science study plan to become a software engineer.☆11Feb 17, 2024Updated 2 years ago
- Implementation of RegNetY in TensorFlow 2☆21Jan 15, 2023Updated 3 years ago
- ☆12Jan 14, 2023Updated 3 years ago
- Wordpress hosting with auto-scaling - Free Trial Offer • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- This repository showcases a collection of machine learning projects in various domains, demonstrating my skills and expertise as a data s…☆12Nov 20, 2023Updated 2 years ago
- ☆16Aug 5, 2023Updated 2 years ago
- ☆19May 11, 2023Updated 3 years ago
- Python wrapper for Goodreads API☆30Feb 20, 2020Updated 6 years ago
- With everything I learned from DEZoomcamp from datatalks.club, this project performs a batch processing on AWS for the cycling dataset wh…☆15Jan 4, 2026Updated 6 months ago
- Dockerized analytics pipeline where Airflow moves mock-API data into Postgres, runs dbt transformations, and serves clean tables for Powe…☆80Oct 18, 2025Updated 9 months ago
- ☆33Jul 29, 2023Updated 2 years ago