A Data Engineering Project that implements an ETL data pipeline using Dagster, Apache Spark, Streamlit, MinIO, Metabase, Dbt, Polars, Docker. Data from kaggle and youtube-api
☆22Nov 19, 2024Updated last year
Alternatives and similar repositories for Youtube-Recommend-Master-ETL-Pipeline
Users that are interested in Youtube-Recommend-Master-ETL-Pipeline are comparing it to the libraries listed below
Sorting:
- This project implements an ELT (Extract - Load - Transform) data pipeline with the goodreads dataset, using dagster (orchestration), spar…☆43Apr 22, 2023Updated 2 years ago
- Scan and monitor your network effortlessly! Nmap Prometheus Exporter provides insights into network health and security with Prometheus-c…☆15Oct 2, 2023Updated 2 years ago
- A quickstart tool for creating a FastAPI project with Jinja2, TailwindCSS, Flowbite, HTMX, and AlpineJS.☆13Jun 23, 2025Updated 8 months ago
- A collection of data engineering projects: data modeling, ETL pipelines, data lakes, infrastructure configuration on AWS, data warehousin…☆15Apr 29, 2021Updated 4 years ago
- velib-v2: An ETL pipeline that employs batch and streaming jobs using Spark, Kafka, Airflow, and other tools, all orchestrated with Docke…☆20Aug 12, 2025Updated 6 months ago
- End-to-end ELT data engineering project☆22Dec 24, 2022Updated 3 years ago
- DataTalks.Club's Data Engineering Zoomcamp Project☆24Jul 14, 2022Updated 3 years ago
- A end-to-end real-time stock market data pipeline with Python, AWS EC2, Apache Kafka, and Cassandra Data is processed on AWS EC2 with Apa…☆28Jun 7, 2023Updated 2 years ago
- Study and research with your docs, media, and AI in one place☆34Updated this week
- Simple ETL pipeline using Python☆29May 22, 2023Updated 2 years ago
- Telegram bot that manages creation of queues / attendance lists for periodic events.☆15Dec 23, 2024Updated last year
- ☆16Feb 20, 2026Updated 2 weeks ago
- a tui ssh app framework for rust☆16Feb 27, 2026Updated last week
- This project aims to build a traveling recommendation application using Google Places API and OpenAI LLM.☆11Mar 19, 2024Updated last year
- ☆41Mar 9, 2025Updated last year
- Simple terminal UI for managing /etc/hosts files!☆17Aug 18, 2024Updated last year
- GraphQL API for cloud pricing. Contains over 3M public prices from AWS, Azure and GCP. Self-updates prices via an automated weekly job.☆19Updated this week
- Feature Packed Self-Hosted File Sharing☆11Aug 14, 2025Updated 6 months ago
- 🥢 The simplest way to create REST API with Node.js, Express.js, and TypeORM.☆11Oct 30, 2025Updated 4 months ago
- csv to parquet and vice versa file converter based on Pandas written in Python3☆10Mar 23, 2021Updated 4 years ago
- Program summarizes news articles into a couple of sentences. This project was inspired by SMMRY, the algorithm used in many subreddits to…☆10Jan 15, 2019Updated 7 years ago
- Tools for diffing and comparing web content. Also includes a web server that makes diffs available as an HTTP service.☆18Mar 1, 2026Updated last week
- Zero-Config Terraform Module to deploy Next.js Apps on AWS using Serverless resources☆12Dec 4, 2025Updated 3 months ago
- OpenExchange web application☆16Apr 5, 2025Updated 11 months ago
- A simple javascript library for reading glucose data from the Dexcom Share API.☆10Aug 11, 2023Updated 2 years ago
- My solutions for the Udacity Data Engineering Nanodegree☆34Oct 14, 2019Updated 6 years ago
- An example of a Dagster project with a possible folder structure to organize the assets, jobs, repositories, schedules, and ops. Also has…☆102Nov 3, 2024Updated last year
- ☆11Nov 9, 2022Updated 3 years ago
- terrajux diffs the source code of a terraform project and all of its transitive module dependencies between two git refs.☆11Jun 14, 2021Updated 4 years ago
- The repository includes detailed steps to get data from GES DISC, convert HDF5 files to CSV and plotting geographic data.☆11Aug 17, 2020Updated 5 years ago
- ☆11Feb 27, 2024Updated 2 years ago
- A python web Application Scaffolding tool Inspired by the Laravel artisan tool☆17Feb 9, 2026Updated last month
- Langflow chat proxy and frontend using FastAPI and HTMX☆16Jul 11, 2024Updated last year
- 📕Ansible playbooks for Raspberry Pi, Linux and Mac☆14Dec 22, 2024Updated last year
- ryantoken.com v3 - written in Svelte, Tailwind, and mdsvex. Replaced by ryantoken.com v4☆12Jul 27, 2025Updated 7 months ago
- Tool to scan a python based repo and outline a text based report used for LLMs☆13Feb 19, 2024Updated 2 years ago
- Streamlit Dashboard over Superstore Data stored in Postgres Docker container. With SQLAlchemy + Plotly Express☆13Oct 16, 2024Updated last year
- A Python Snowpark CLI for loading the TPC-DI dataset into Snowflake. Additional dbt models for building the data warehouse.☆10Sep 4, 2025Updated 6 months ago
- Fritz!Box tool set, e.g. parsing the call monitor, adding phone entries to phonebook, auto-blocking calls etc.☆11Feb 19, 2023Updated 3 years ago