An end-to-end data engineering pipeline that fetches data from Wikipedia, cleans and transforms it with Apache Airflow and saves it on Azure Data Lake. Other processing takes place on Azure Data Factory, Azure Synapse and Tableau.
☆30Oct 2, 2023Updated 3 years ago
Alternatives and similar repositories for FootballDataEngineering
Users that are interested in FootballDataEngineering are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- This project provides an end-to-end data processing and visualization of visa numbers in Japan using PySpark and Plotly. The spark cluste…☆11Oct 11, 2023Updated 3 years ago
- This project serves as a comprehensive guide to building an end-to-end data engineering pipeline using TCP/IP Socket, Apache Spark, OpenA…☆47Jan 4, 2024Updated 2 years ago
- In this project, we setup and end to end data engineering using Apache Spark, Azure Databricks, Data Build Tool (DBT) using Azure as our …☆40Dec 18, 2023Updated 2 years ago
- An end-to-end data engineering pipeline that fetches real-time YouTube analytics and streams them through Kafka for processing with ksqlD…☆16Sep 19, 2023Updated 3 years ago
- This project provides a comprehensive data pipeline solution to extract, transform, and load (ETL) Reddit data into a Redshift data wareh…☆229Oct 23, 2023Updated 2 years ago
- Bare Metal GPUs on DigitalOcean Gradient AI • AdPurpose-built for serious AI teams training foundational models, running large-scale inference, and pushing the boundaries of what's possible.
- A data pipeline for processing football data using Python and SQL☆13Sep 12, 2023Updated 3 years ago
- using pandas to analyze data from brazilian federal senate☆10Nov 30, 2017Updated 8 years ago
- Transform data from on-premises SQL Server to Azure Delta Lake Storage for Analytics and Visualization☆29Jul 16, 2023Updated 3 years ago
- This project shows how to capture changes from postgres database and stream them into kafka☆44May 17, 2024Updated 2 years ago
- Produce Kafka messages, consume them and upload into Cassandra, MongoDB.☆43Sep 26, 2023Updated 3 years ago
- This repository contains an Apache Flink application for real-time sales analytics built using Docker Compose to orchestrate the necessar…☆53Dec 4, 2023Updated 2 years ago
- ☆47Jul 6, 2024Updated 2 years ago
- ☆11Aug 8, 2023Updated 3 years ago
- This project demonstrates how to use Apache Airflow to submit jobs to Apache spark cluster in different programming laguages using Python…☆47Mar 14, 2024Updated 2 years ago
- 1-Click AI Models by DigitalOcean Gradient • AdDeploy popular AI models on DigitalOcean Gradient GPU virtual machines with just a single click. Zero configuration with optimized deployments.
- Quantum Black Hackathon organised by Analytics Vidya☆13Jul 23, 2019Updated 7 years ago
- ms-dataverse is a Python module for Microsoft Dataverse, offering a lightweight ORM to query, create, update, and delete entities. Utiliz…☆13Apr 10, 2023Updated 3 years ago
- ☆17Aug 2, 2025Updated last year
- Snowflake - Build and Architect Data Pipelines using AWS, published by Packt☆24Apr 3, 2023Updated 3 years ago
- The official documentation of the City of Boston's Analytics Team.☆13Jan 21, 2025Updated last year
- Create a streaming data, transfer it to Kafka, modify it with PySpark, take it to ElasticSearch and MinIO☆66Jul 21, 2023Updated 3 years ago
- Site da Python Brasil 2020☆11Nov 6, 2020Updated 5 years ago
- End-to-end ETL pipeline in the Microsoft Azure cloud - (Jun '24 - Jul '24)☆41Aug 9, 2024Updated 2 years ago
- Resources for tracking DP-700 exam prep progress.☆59Jun 14, 2025Updated last year
- AI Agents on DigitalOcean Gradient AI Platform • AdBuild production-ready AI agents using customizable tools or access multiple LLMs through a single endpoint. Create custom knowledge bases or connect external data.
- ☆18Feb 1, 2025Updated last year
- Data pipeline from device to cloud☆10May 14, 2022Updated 4 years ago
- ☆20Feb 14, 2025Updated last year
- Toolset for detecting reflected xss in websites☆16Oct 6, 2018Updated 8 years ago
- Local SQL Database ---> Azure ---> Power BI☆15Oct 13, 2023Updated 2 years ago
- ☆12Jan 14, 2023Updated 3 years ago
- Welcome to a fantastic place to start learning to code! This repo is dedicated to making the learning process easy, fun, and enjoyable!☆16Jan 26, 2021Updated 5 years ago
- ☆16Aug 5, 2023Updated 3 years ago
- ☆19May 11, 2023Updated 3 years ago
- Managed Database hosting by DigitalOcean • AdPostgreSQL, MySQL, MongoDB, Kafka, Valkey, and OpenSearch available. Automatically scale up storage and focus on building your apps.
- Functional Data Engineering tutorial in Python & Airflow.☆17Mar 24, 2023Updated 3 years ago
- End-to-end Data Project (DA/DS/DE/MLOps) - retail/e-commerce - interpretable dynamic clustering☆23Jul 12, 2025Updated last year
- With everything I learned from DEZoomcamp from datatalks.club, this project performs a batch processing on AWS for the cycling dataset wh…☆15Jan 4, 2026Updated 9 months ago
- Python wrapper for Goodreads API☆30Feb 20, 2020Updated 6 years ago
- Dockerized analytics pipeline where Airflow moves mock-API data into Postgres, runs dbt transformations, and serves clean tables for Powe…☆85Oct 18, 2025Updated 11 months ago
- ☆33Jul 29, 2023Updated 3 years ago
- ☆14Oct 1, 2022Updated 4 years ago