On-car / spotify-end-to-end-data-engineering--project
Integrating with Spotify API and extracting Data. Deploying code on AWS Lambda for Data Extraction. Adding trigger to run the extraction automatically Writing transformation function. Building automated trigger on transformation function. Store files on S3 properly. Building Analytics Tables on data files using Glue and Athena.
☆10Updated last year
Alternatives and similar repositories for spotify-end-to-end-data-engineering--project:
Users that are interested in spotify-end-to-end-data-engineering--project are comparing it to the libraries listed below
- This project demonstrates how to build and automate an ETL pipeline written in Python and schedule it using open source Apache Airflow or…☆14Updated 2 weeks ago
- ☆12Updated 2 years ago
- ☆30Updated 4 months ago
- YouTube tutorial project☆101Updated last year
- ☆11Updated last year
- This project focuses on building a robust data pipeline using Apache Airflow to automate the ingestion of weather data from the OpenWeath…☆21Updated last year
- ☆18Updated last year
- Saturn Cloud starter code for LLM Zoomcamp☆11Updated 9 months ago
- An end-to-end data engineering pipeline that fetches real-time YouTube analytics and streams them through Kafka for processing with ksqlD…☆11Updated last year
- In this project, we setup and end to end data engineering using Apache Spark, Azure Databricks, Data Build Tool (DBT) using Azure as our …☆27Updated last year
- mlops main demo☆15Updated 2 years ago
- ☆64Updated this week
- code snippet for analytics sessions☆34Updated 2 years ago
- This repository will help you to learn about databricks concept with the help of examples. It will include all the important topics which…☆98Updated 8 months ago
- ☆13Updated last year
- Foundational Training on MlFlow and Data Science on Databricks.☆17Updated 2 years ago
- ☆32Updated last year
- Databricks ML in Action, Published by Packt☆30Updated 11 months ago
- I am using confluent Kafka cluster to produce and consume scraped data. In this project, I've created a real-time data pipeline that uti…☆29Updated last year
- Databricks Certified Associate Spark Developer preparation toolkit to setup single node Standalone Spark Cluster along with material in t…☆30Updated last year
- Datastructure for data science☆22Updated last year
- In this project, we will build and ETL(Extract,Transform,Load) pipeline using the Spotify API on AWS. The pipeline will retrieve data fro…☆21Updated last year
- Learn AWS Automation with boto3, Python, and Lambda Functions, by Packt Publishing☆16Updated 2 years ago
- Resources for the free AWS Data Engineering course on youtube☆99Updated 3 years ago
- ☆12Updated 7 months ago
- Snowflake - Build and Architect Data Pipelines using AWS, published by Packt☆20Updated 2 years ago
- ☆15Updated last year
- ☆25Updated 2 years ago
- PySpark Cheatsheet☆36Updated 2 years ago
- ☆29Updated 6 months ago