Joshua-omolewa / Retailstore_ETL_pipeline_project
Built a Data Pipeline for a Retail store using AWS services that collects data from its transactional database (OLTP) in Snowflake and transforms the raw data (ETL process) using Apache spark to meet business requirements and also enables Data Analyst create Data Visualization using Superset. Airflow is used to orchestrate the pipeline
☆9Updated last year
Alternatives and similar repositories for Retailstore_ETL_pipeline_project:
Users that are interested in Retailstore_ETL_pipeline_project are comparing it to the libraries listed below
- 😈Complete End to End ETL Pipeline with Spark, Airflow, & AWS☆45Updated 5 years ago
- In this project, we will build and ETL(Extract,Transform,Load) pipeline using the Spotify API on AWS. The pipeline will retrieve data fro…☆21Updated last year
- ☆21Updated last year
- ☆28Updated last year
- Simple ETL pipeline using Python☆26Updated last year
- YouTube tutorial project☆102Updated last year
- End to end data engineering project with kafka, airflow, spark, postgres and docker.☆90Updated 3 weeks ago
- Ravi Azure ADB ADF Repository☆66Updated 2 months ago
- Classwork projects and home works done through Udacity data engineering nano degree☆74Updated last year
- Big Data Engineering practice project, including ETL with Airflow and Spark using AWS S3 and EMR☆82Updated 5 years ago
- ☆33Updated last year
- Resources and projects from Udacity Data Engineering with AWS nano degree programme☆25Updated 2 years ago
- A collection of data engineering projects: data modeling, ETL pipelines, data lakes, infrastructure configuration on AWS, data warehousin…☆15Updated 3 years ago
- ☆23Updated 2 years ago
- Ultimate guide for mastering Spark Performance Tuning and Optimization concepts and for preparing for Data Engineering interviews☆118Updated 10 months ago
- This project focuses on building a robust data pipeline using Apache Airflow to automate the ingestion of weather data from the OpenWeath…☆21Updated last year
- ☆87Updated 2 years ago
- PySpark Projects☆23Updated last week
- ☆40Updated 9 months ago
- Repository for Data Engineering Interview Series☆29Updated 5 months ago
- PySpark functions and utilities with examples. Assists ETL process of data modeling☆101Updated 4 years ago
- Projects done in the Data Engineer Nanodegree Program by Udacity.com☆158Updated 2 years ago
- This is the final project that after participated the Data Engineering Zoomcamp☆11Updated 3 years ago
- Udacity Data Engineering Nanodegree Capstone Project☆36Updated 4 years ago
- Solution to all projects of Udacity's Data Engineering Nanodegree: Data Modeling with Postgres & Cassandra, Data Warehouse with Redshift,…☆56Updated 2 years ago
- A batch processing data pipeline, using AWS resources (S3, EMR, Redshift, EC2, IAM), provisioned via Terraform, and orchestrated from loc…☆22Updated 2 years ago
- Data pipeline performing ETL to AWS Redshift using Spark, orchestrated with Apache Airflow☆142Updated 4 years ago
- Step by step instructions to create a production-ready data pipeline☆44Updated 3 months ago
- All Data Engineering notebooks from Datacamp course☆115Updated 5 years ago
- This repository contains the code for a realtime election voting system. The system is built using Python, Kafka, Spark Streaming, Postgr…☆36Updated last year