rafik-rahoui / End-to-end-data-enginnerring-project
End-to-end ELT data engineering project
☆20Updated last year
Related projects ⓘ
Alternatives and complementary repositories for End-to-end-data-enginnerring-project
- DataTalks.Club's Data Engineering Zoomcamp Project☆22Updated 2 years ago
- ☆11Updated 2 years ago
- ☆16Updated last year
- ☆27Updated last year
- In this project, we will build and ETL(Extract,Transform,Load) pipeline using the Spotify API on AWS. The pipeline will retrieve data fro…☆21Updated last year
- A simple pipeline utilising cron, Postgres, AWS EC2, and Metabase☆10Updated 4 months ago
- ☆18Updated 10 months ago
- ☆38Updated 4 months ago
- This repository contains the code for a realtime election voting system. The system is built using Python, Kafka, Spark Streaming, Postgr…☆32Updated 11 months ago
- Simple ETL pipeline using Python☆21Updated last year
- This project focuses on building a robust data pipeline using Apache Airflow to automate the ingestion of weather data from the OpenWeath…☆21Updated last year
- A collection of data engineering projects: data modeling, ETL pipelines, data lakes, infrastructure configuration on AWS, data warehousin…☆14Updated 3 years ago
- End to end data engineering project☆51Updated 2 years ago
- A project portfolio to accompany my resume☆22Updated last year
- Example repo to create end to end tests for data pipeline.☆21Updated 5 months ago
- Projects done in the Data Engineer Nanodegree Program by Udacity.com☆94Updated last year
- A end-to-end real-time stock market data pipeline with Python, AWS EC2, Apache Kafka, and Cassandra Data is processed on AWS EC2 with Apa…☆22Updated last year
- Data Engineering Project in GCP☆19Updated last year
- My solutions for the Udacity Data Engineering Nanodegree☆33Updated 5 years ago
- Code for "Advanced data transformations in SQL" free live workshop☆67Updated last month
- Create a streaming data, transfer it to Kafka, modify it with PySpark, take it to ElasticSearch and MinIO☆57Updated last year
- Writes the CSV file to Postgres, read table and modify it. Write more tables to Postgres with Airflow.☆35Updated last year
- Sample project to demonstrate data engineering best practices☆166Updated 9 months ago
- An end-to-end data engineering pipeline to create a dashboard for the latest content on the r/Stocks subreddit☆19Updated 2 years ago
- Get Crypto data from API, stream it to Kafka with Airflow. Write data to MySQL and visualize with Metabase☆13Updated last year
- Data Engineering Project to Extract and Process Solana Reddit Data☆23Updated 9 months ago
- ☆86Updated 2 years ago
- velib-v2: An ETL pipeline that employs batch and streaming jobs using Spark, Kafka, Airflow, and other tools, all orchestrated with Docke…☆18Updated 2 months ago
- 😈Complete End to End ETL Pipeline with Spark, Airflow, & AWS☆43Updated 5 years ago
- End to end data engineering project with kafka, airflow, spark, postgres and docker.☆67Updated 3 months ago