This is a comprehensive end-to-end data engineering project. I extracted data directly from YouTube in raw JSON format using Python and AWS Lambda, performed transformations using Apache Spark and AWS Glue, and finally loaded the data into a table format in the data warehouse using Snowflake.
☆11Jun 4, 2024Updated last year
Alternatives and similar repositories for Youtube_End_To_End_Data_Pipeline
Users that are interested in Youtube_End_To_End_Data_Pipeline are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- ☆12Mar 17, 2022Updated 4 years ago
- Codebasics Resume Project Challenge, Provide Insights to Revenue Team in Hospitality Domain☆10Sep 27, 2022Updated 3 years ago
- code-snippets☆13Oct 22, 2025Updated 5 months ago
- This repository contains the "RFM Analysis" for a Sales Data of a Retailer in SQL. This is part of my Data Science Portfolio Projects☆10Jul 3, 2023Updated 2 years ago
- The goal of this project is to analyse the impact of Covid-19 on the Aviation industry through data engineering processes using technolog…☆13Jun 26, 2022Updated 3 years ago
- End-to-end encrypted cloud storage - Proton Drive • AdSpecial offer: 40% Off Yearly / 80% Off First Month. Protect your most important files, photos, and documents from prying eyes.
- Picarch is a Python project for face detection and image similarity search using insightface and PostgreSQL.☆14Apr 8, 2025Updated 11 months ago
- A t-sne implementation on GOT dataset☆12Jan 4, 2024Updated 2 years ago
- 🐍🗺️ This Python script empowers you to scrape data from Google Maps, enabling extraction of valuable information like addresses, review…☆12Aug 5, 2023Updated 2 years ago
- Complete Azure Data Factory CICD Process Via Azure Pipeline☆26Jan 31, 2024Updated 2 years ago
- Repository for Spark using Python material. It is popularly known as PySpark.☆20Aug 18, 2021Updated 4 years ago
- This project would demonstrate the following capabilities: 1. Extraction Loading and Transformation of S&P 500 data and company fundament…☆14Sep 26, 2021Updated 4 years ago
- data-warehouse-snowflake-for-data-engineering☆19Sep 14, 2023Updated 2 years ago
- This is Project Repository For Password Generator Project In Web Development 2.0 - iNeuron☆12Mar 14, 2024Updated 2 years ago
- A posenet demo built using ml5.js☆16Feb 7, 2023Updated 3 years ago
- Open source password manager - Proton Pass • AdSecurely store, share, and autofill your credentials with Proton Pass, the end-to-end encrypted password manager trusted by millions.
- This repository contains tasks on how to build an ETL pipeline for the online transaction data of an e-commerce company.☆18Jun 27, 2023Updated 2 years ago
- Tableau Projects for data analysis, data analytics and data visualaization on different data sets☆16Jun 18, 2020Updated 5 years ago
- My solutions to exercises from various SQL learning courses and platforms☆12Jun 23, 2021Updated 4 years ago
- An API based NLP application created using Tkinter and OOP☆13Dec 3, 2022Updated 3 years ago
- JP morgan virtual internship Quantitative Research☆22Dec 24, 2023Updated 2 years ago
- ☆17Jan 29, 2023Updated 3 years ago
- This project provides Inventory Management using Power BI, extremely useful for Warehouse/ In-plant Inventory Managers to effectively con…☆13Feb 18, 2024Updated 2 years ago
- Python pandas tutorials from Corey Schafer☆25Apr 8, 2023Updated 2 years ago
- ☆13Mar 27, 2018Updated 8 years ago
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting with the flexibility to host WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Cloudways by DigitalOcean.
- ☆13Jan 2, 2024Updated 2 years ago
- Super Mario is a legendary game we all cherish! In this project, we will deploy Super Mario on Amazon EKS (Elastic Kubernetes Service) us…☆11Feb 3, 2026Updated last month
- pandas I/O functions☆10Jan 23, 2023Updated 3 years ago
- simple viz tools to visualize matrix linear transformation☆16Apr 15, 2023Updated 2 years ago
- INSAID Assignment to create a ML model to detect fraud transactions for a financial company.☆16Nov 19, 2022Updated 3 years ago
- Contains tools for analyzing time-series data.☆11May 8, 2013Updated 12 years ago
- Indago is a web-based job tracking application that allows job seekers to easily keep track of their job search progress.☆11Dec 8, 2024Updated last year
- This project focuses on creating comprehensive financial reports using Power BI, leveraging Data Analysis Expressions (DAX), Excel data, …☆16Dec 24, 2023Updated 2 years ago
- A sample python web app using SQL and Streamlit☆15Sep 30, 2023Updated 2 years ago
- DigitalOcean Gradient AI Platform • AdBuild production-ready AI agents using customizable tools or access multiple LLMs through a single endpoint. Create custom knowledge bases or connect external data.
- This repository provides various web scraping projects in Jupyter notebooks for both learning and data-related workshopes☆13Oct 21, 2022Updated 3 years ago
- Find all ExcelR Data Analyst Assignment Solution Here 1. Advanced Excel 2. MySQL 3. Python 4. Tableau 5. Power BI☆15Mar 3, 2024Updated 2 years ago
- This project contain build end-to-end e-commerce data from data source into data warehouse and visualization.☆13Sep 5, 2024Updated last year
- Deploying Architecture in AWS Using Terraform as Infrastructure as Code☆18Jul 13, 2024Updated last year
- Free open-source course for demonstrating Power BI Desktop capabilities under MIT license.☆21May 22, 2017Updated 8 years ago
- Generates and updates a human-readable description for a Postman Collection using the OpenAI API, based on the collection ID provided as …☆14Mar 30, 2023Updated 2 years ago
- End-to-End BI & DW project: Data Warehousing design and modeling (MySQL), ETL (PDI) and Dashboard (Tableau)☆16Aug 10, 2020Updated 5 years ago