faizeraza / dataengineering-github-data-pipelineline
In this project I have built etl pipline which scraps the trending repository based on month,week and day LIVE extract other related information using API and transform it into star schema and load to postgresql then analyzed it over PowerBI . Deployment repo link:
☆12Updated last year
Alternatives and similar repositories for dataengineering-github-data-pipelineline:
Users that are interested in dataengineering-github-data-pipelineline are comparing it to the libraries listed below
- This data project can be used as a take-home assignment to learn Pyspark and Data Engineering.☆14Updated 2 years ago
- ☆28Updated last year
- In this project, we will build and ETL(Extract,Transform,Load) pipeline using the Spotify API on AWS. The pipeline will retrieve data fro…☆21Updated 2 years ago
- Resources and projects from Udacity Data Engineering with AWS nano degree programme☆25Updated 2 years ago
- DataTalks.Club's Data Engineering Zoomcamp Project☆23Updated 2 years ago
- Capstone Project for the IBM Data Engineering Professional Certification.☆10Updated 3 years ago
- Data Engineering Project in GCP☆20Updated 2 years ago
- This is the final project that after participated the Data Engineering Zoomcamp☆11Updated 3 years ago
- End to end data engineering project☆54Updated 2 years ago
- ☆21Updated last year
- ☆17Updated 2 years ago
- Code Repository for my 3rd Data Project.☆14Updated last year
- End-to-end ELT data engineering project☆21Updated 2 years ago
- A batch Data Pipeline that retrieves data from a user purchase table and a movie review table and is transformed to form a user behaviour…☆16Updated 2 years ago
- ☆12Updated 4 years ago
- DataTalks.Club's Data Engineering Zoomcamp Project☆11Updated 2 years ago
- Simple ETL pipeline using Python☆26Updated last year
- Business challenge that requires building a data platform for retailer data analytics.☆12Updated 2 years ago
- With everything I learned from DEZoomcamp from datatalks.club, this project performs a batch processing on AWS for the cycling dataset wh…☆14Updated 3 years ago
- Data Pipeline from the Global Historical Climatology Network DataSet☆27Updated 2 years ago
- data-warehouse-snowflake-for-data-engineering☆17Updated last year
- velib-v2: An ETL pipeline that employs batch and streaming jobs using Spark, Kafka, Airflow, and other tools, all orchestrated with Docke…☆18Updated 7 months ago
- An end-to-end data engineering pipeline to create a dashboard for the latest content on the r/Stocks subreddit☆19Updated 2 years ago
- Cool DE Projects☆28Updated 2 months ago
- Built a Data Pipeline for a Retail store using AWS services that collects data from its transactional database (OLTP) in Snowflake and tr…☆9Updated last year
- YouTube tutorial project☆101Updated last year
- Simple stream processing pipeline☆102Updated 10 months ago
- Code Repository for my 1st Data Project.☆23Updated 2 years ago
- Ultimate guide for mastering Spark Performance Tuning and Optimization concepts and for preparing for Data Engineering interviews☆127Updated 11 months ago
- A data engineering project with Airflow, dbt, Terrafrom, GCP and much more!☆24Updated 2 years ago