andreichiro / data_engineer_end2end
End-to-end data engineer project
☆16Updated last year
Related projects: ⓘ
- Repository for Data Engineering Zoomcamp 2024☆13Updated 5 months ago
- Code for blog at https://www.startdataengineering.com/post/python-for-de/☆47Updated 3 months ago
- Code for "Advanced data transformations in SQL" free live workshop☆54Updated last month
- ☆29Updated last year
- This repo is meant to make it really easy to analyze the interplays between health and social media use.☆41Updated 2 years ago
- Code for my "Efficient Data Processing in SQL" book.☆47Updated last month
- Data engineering project using UK Bus Open Data Service (BODS) to calculate late buses in real-time for any selected region in England. P…☆26Updated last year
- This project focuses on building a robust data pipeline using Apache Airflow to automate the ingestion of weather data from the OpenWeath…☆20Updated last year
- build dw with dbt☆26Updated last month
- This repo contains all the material developed during the 9-week bootcamp provided by DPhi in colaboration with DataTalks Club☆21Updated 2 years ago
- Cost Efficient Data Pipelines with DuckDB☆42Updated last month
- reating a modern data pipeline using a combination of Terraform, AWS Lambda and S3, Snowflake, DBT, Mage AI, and Dash.☆12Updated last year
- This is a demo streaming project simulating a music streaming service.☆23Updated last month
- DataTalks.Club's Data Engineering Zoomcamp Project☆19Updated 2 years ago
- Sample project to demonstrate data engineering best practices☆156Updated 6 months ago
- ☆35Updated 2 months ago
- ☆12Updated last month
- In this project, we setup and end to end data engineering using Apache Spark, Azure Databricks, Data Build Tool (DBT) using Azure as our …☆19Updated 9 months ago
- A self-contained, ready to run Airflow ELT project. Can be run locally or within codespaces.☆54Updated last year
- Data pipeline that scrapes Rust cheater Steam profiles☆50Updated 2 years ago
- Duke MIDS: Data Engineering and DataOps Course☆55Updated last year
- A modern ELT demo using airbyte, dbt, snowflake and dagster☆22Updated last year
- A list of all my posts and personal projects☆64Updated 3 months ago
- This repository contains the code for a realtime election voting system. The system is built using Python, Kafka, Spark Streaming, Postgr…☆24Updated 9 months ago
- ☆16Updated last year
- This is a public repository to go over all the LLM-driven data engineering concepts.☆67Updated last year
- Demo on how to use Prefect with Docker☆26Updated 2 years ago
- Some recipes for data engineering with Python☆22Updated 3 years ago
- Material for PyData NYC Tutorial on Large Scale Timeseries Forecasting☆25Updated last year
- A batch processing data pipeline, using AWS resources (S3, EMR, Redshift, EC2, IAM), provisioned via Terraform, and orchestrated from loc…☆20Updated 2 years ago