Generate synthetic Spotify music stream dataset to create dashboards. Spotify API generates fake event data emitted to Kafka. Spark consumes and processes Kafka data, saving it to the Datalake. Airflow orchestrates the pipeline. dbt moves data to Snowflake, transforms it, and creates dashboards.
☆71Dec 17, 2023Updated 2 years ago
Alternatives and similar repositories for spotify-stream-analytics
Users that are interested in spotify-stream-analytics are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- A data pipeline with Kafka, Spark Streaming, dbt, Docker, Airflow, and GCP!☆12Jul 6, 2023Updated 3 years ago
- velib-v2: An ETL pipeline that employs batch and streaming jobs using Spark, Kafka, Airflow, and other tools, all orchestrated with Docke…☆20Aug 12, 2025Updated last year
- ☆13Oct 15, 2021Updated 5 years ago
- ⚙️ Airflow data pipeline with Terraform, GCP BigQuery, dbt, Soda and Looker Studio.☆26Oct 19, 2023Updated 3 years ago
- Implementation of a Q&A Chatbot with RAG using the Data Engineering Community WhatsApp group messages.☆10Jan 17, 2025Updated last year
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- Local development environment for python data projects, with Docker☆22Dec 14, 2022Updated 3 years ago
- ☆10Nov 17, 2021Updated 5 years ago
- Examples of Apache Flink® v2.1 applications showcasing the DataStream API, Table API in Java and Python, and Flink SQL, featuring AWS, Gi…☆10Jan 13, 2026Updated 10 months ago
- Testing Spark Structured Streaming anf Kafka with real data from traffic sensors☆17Nov 11, 2022Updated 4 years ago
- This repo consists of all important concepts for data engineers.☆11Jun 2, 2026Updated 5 months ago
- This project is for demonstrating knowledge of Data Engineering tools and concepts and also learning in the process☆43Dec 1, 2022Updated 3 years ago
- Ease With Data☆11Mar 19, 2023Updated 3 years ago
- A simple and easy to use Data Quality (DQ) tool built with Python.☆51Sep 7, 2023Updated 3 years ago
- ☆15Apr 4, 2026Updated 7 months ago
- Managed Database hosting by DigitalOcean • AdPostgreSQL, MySQL, MongoDB, Kafka, Valkey, and OpenSearch available. Automatically scale up storage and focus on building your apps.
- Autonomous UAV-Based Maritime SAR using onboard YOLOv8/v11 for real-time swimmer detection and precision life buoy deployment. Combines G…☆17Aug 7, 2026Updated 3 months ago
- Using Apache Spark SQL, Spark ML, Pandas to analyse and predict using the Chicago crime dataset☆10Apr 6, 2018Updated 8 years ago
- The goal of this project is to build an ETL pipeline. The data would be processed as a batch (monthly) between 2018-01 and 2021-02.☆14Mar 26, 2022Updated 4 years ago
- ☆13Mar 20, 2025Updated last year
- ☆14Mar 1, 2026Updated 8 months ago
- End-to-end data platform leveraging the Modern data stack☆53Apr 10, 2024Updated 2 years ago
- Analysis of New York State Police Department Arrests dataset. Created Dimensional Model for the provided dataset. Using Alteryx and Talen…☆19Sep 19, 2022Updated 4 years ago
- A dockerized 5 min. mini-batch data pipeline. Spark Structured Streaming: reading from Kafka to PostgreSQL as a sink DB. Airflow, Kafka, …☆10Jul 17, 2024Updated 2 years ago
- ☆15May 13, 2024Updated 2 years ago
- Deploy open-source AI quickly and easily - Special Bonus Offer • AdRunpod Hub is built for open source. One-click deployment and autoscaling endpoints without provisioning your own infrastructure.
- In this article, you will learn how to set up a real-time data processing and analytics environment using Docker, MySQL, Redpanda, MinIO,…☆11Jun 27, 2023Updated 3 years ago
- This project is built for personal purposes. Currently, I am enahancing it with an interactive bot that can recommend and help you improv…☆26Jul 27, 2026Updated 3 months ago
- commit-linter helps you lint your commit messages according to commit conventions☆10Aug 17, 2023Updated 3 years ago
- 📦 Starting box for Vagrant. Inside box Ubuntu 20.04 LTS with Git, Docker and Docker compose.☆19May 5, 2022Updated 4 years ago
- Building a Comprehensive Repository of Hromada-Level Data in Ukraine to Facilitate Research and Informed Policy Decisions. This repositor…☆17Nov 6, 2026Updated 2 weeks ago
- Template for Retreival-Augmented Generation (RAG) application using Milvus for vector storage, LangGraph for ReAct agentic workflows, and…☆12May 21, 2025Updated last year
- (Python, PySpark)☆10Nov 15, 2020Updated 6 years ago
- ☆10Sep 21, 2021Updated 5 years ago
- DevOps☆16May 17, 2021Updated 5 years ago
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- Archive of Fortran code for various atmosphere models from http://www.dem.inpe.br/~val/atmod/default.html☆11Dec 14, 2019Updated 6 years ago
- Case Study's from Danny Ma's Serious SQL Course☆19Aug 4, 2022Updated 4 years ago
- ☆16May 29, 2023Updated 3 years ago
- Tools to simplify life with AI☆29Apr 4, 2025Updated last year
- Data pipeline performing ETL to AWS Redshift using Spark, orchestrated with Apache Airflow☆167Jun 16, 2020Updated 6 years ago
- This is a simple object detection application, which can be used to detect military objects☆15Nov 16, 2020Updated 6 years ago
- ☆26Jan 31, 2023Updated 3 years ago
Are these results useful?