faizeraza / dataengineering-github-data-pipelinelineLinks
In this project I have built etl pipline which scraps the trending repository based on month,week and day LIVE extract other related information using API and transform it into star schema and load to postgresql then analyzed it over PowerBI . Deployment repo link:
☆12Updated last year
Alternatives and similar repositories for dataengineering-github-data-pipelineline
Users that are interested in dataengineering-github-data-pipelineline are comparing it to the libraries listed below
Sorting:
- This is the final project that after participated the Data Engineering Zoomcamp☆11Updated 3 years ago
- Resources and projects from Udacity Data Engineering with AWS nano degree programme☆25Updated 2 years ago
- Data Engineering Project in GCP☆20Updated 2 years ago
- In this project, we will build and ETL(Extract,Transform,Load) pipeline using the Spotify API on AWS. The pipeline will retrieve data fro…☆23Updated 2 years ago
- Projects done in the Data Engineer Nanodegree Program by Udacity.com☆160Updated 2 years ago
- YouTube tutorial project☆105Updated last year
- Capstone Project for the IBM Data Engineering Professional Certification.☆10Updated 3 years ago
- Ultimate guide for mastering Spark Performance Tuning and Optimization concepts and for preparing for Data Engineering interviews☆152Updated last year
- ☆28Updated last year
- ☆151Updated 3 years ago
- End to end data engineering project with kafka, airflow, spark, postgres and docker.☆98Updated 3 months ago
- A collection of data engineering projects: data modeling, ETL pipelines, data lakes, infrastructure configuration on AWS, data warehousin…☆15Updated 4 years ago
- This is a template you can use for your next data engineering portfolio project.☆179Updated 3 years ago
- For the Coursera specialization https://www.coursera.org/specializations/gcp-data-machine-learning☆94Updated 7 years ago
- Sample project to demonstrate data engineering best practices☆194Updated last year
- End to end data engineering project☆57Updated 2 years ago
- Data Engineering YouTube Analysis Project by Darshil Parmar☆197Updated last year
- ☆21Updated last year
- Data pipeline performing ETL to AWS Redshift using Spark, orchestrated with Apache Airflow☆147Updated 5 years ago
- A template repository to create a data project with IAC, CI/CD, Data migrations, & testing☆266Updated last year
- ☆17Updated 2 years ago
- This data project can be used as a take-home assignment to learn Pyspark and Data Engineering.☆15Updated 2 years ago
- ☆201Updated last year
- Data Engineering Project with Hadoop HDFS and Kafka☆113Updated last year
- Airflow & DBT Cloud Integrated Project Presented at Lagos DBT Community Meetup & DataFestAfrica 23☆13Updated last year
- ☆142Updated 2 years ago
- Stream processing pipeline from Finnhub websocket using Spark, Kafka, Kubernetes and more☆351Updated last year
- FInal project for data zoom camp 2024☆18Updated last year
- ☆282Updated 11 months ago
- This project provides a comprehensive data pipeline solution to extract, transform, and load (ETL) Reddit data into a Redshift data wareh…☆146Updated last year