Wittline / data-engineering-challenge-th
Dockerizing a Python Script for Web Scraping and consume the scraped data using FastApi (www.metroscubicos.com)
☆14Updated 2 years ago
Related projects ⓘ
Alternatives and complementary repositories for data-engineering-challenge-th
- This project is for demonstrating knowledge of Data Engineering tools and concepts and also learning in the process☆44Updated last year
- Challenge Data Engineer☆25Updated 2 years ago
- Project for "Data pipeline design patterns" blog.☆41Updated 3 months ago
- Learn the entire ETL process based on Spotify API data☆247Updated 3 years ago
- End to end data engineering project☆49Updated 2 years ago
- Grupo de estudio Apache Airflow organizado por la comunidad Data Engineering Latam☆15Updated 10 months ago
- A template repository to create a data project with IAC, CI/CD, Data migrations, & testing☆237Updated 4 months ago
- Price Crawler - Tracking Price Inflation☆182Updated 4 years ago
- Data pipeline performing ETL to AWS Redshift using Spark, orchestrated with Apache Airflow☆133Updated 4 years ago
- Streaming data from a transactional database to a data warehouse using Kafka (Confluent Cloud), Snowflake, and PostgreSQL.☆13Updated last year
- Template for Data Engineering and Data Pipeline projects☆104Updated last year
- Produce Kafka messages, consume them and upload into Cassandra, MongoDB.☆37Updated last year
- ☆25Updated 3 years ago
- Public source code for the Udemy online course Apache Airflow: Complete Hands-On Beginner to Advanced Class.☆62Updated 4 years ago
- Pipeline that extracts data from the Spotify API to build a more detailed version of Spotify Wrapped☆31Updated 11 months ago
- Python ETL demo for Hackforge☆31Updated last year
- Near real time ETL to populate a dashboard.☆70Updated 4 months ago
- ☆128Updated 2 years ago
- The goal of this project is to track the expenses of Uber Rides and Uber Eats through data Engineering processes using technologies such …☆116Updated 2 years ago
- ☆111Updated last month
- A data engineering project (Twitter monitor app)☆76Updated 2 years ago
- End to end data engineering project with kafka, airflow, spark, postgres and docker.☆65Updated 3 months ago
- Data Engineering with Google Cloud Platform, published by Packt☆109Updated last year
- Backup for NYC TLC data for the DE Zoomcamp course☆150Updated 2 years ago
- Unit testing using databricks connect☆30Updated 3 years ago
- Educational project on how to build an ETL (Extract, Transform, Load) data pipeline, orchestrated with Airflow.☆296Updated 2 years ago
- Get data from API, run a scheduled script with Airflow, send data to Kafka and consume with Spark, then write to Cassandra☆128Updated last year
- Sample project to demonstrate data engineering best practices☆164Updated 8 months ago
- ☆30Updated last year
- Data engineering with dbt, published by Packt☆60Updated 7 months ago