Welcome to my data engineering projects repository! Here you will find a collection of data engineering projects that I have worked on.
☆25Apr 27, 2023Updated 3 years ago
Alternatives and similar repositories for data-engineering-projects
Users that are interested in data-engineering-projects are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- A Data Engineering Project that implements an ETL data pipeline using Dagster, Apache Spark, Streamlit, MinIO, Metabase, Dbt, Polars, Doc…☆25Nov 19, 2024Updated last year
- Construindo Pipeline de Dados com Astro Python SDK, dbt & Apache Airflow☆10Mar 20, 2024Updated 2 years ago
- ☆13Mar 5, 2023Updated 3 years ago
- In this project I have built etl pipline which scraps the trending repository based on month,week and day LIVE extract other related info…☆12Sep 9, 2023Updated 2 years ago
- 🌟 An end-to-end full-stack data science project, including modelling, MLOps, and data storytelling. ✨☆16Aug 30, 2025Updated last year
- Wordpress hosting with auto-scaling - Free Trial Offer • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- An ETL pipeline for Covid19 Data collected from John Hopkin's Github☆11Dec 18, 2020Updated 5 years ago
- Business Intelligence and Data Warehousing Project☆13Dec 4, 2019Updated 6 years ago
- Spark Structured Streaming data pipeline that processes movie ratings data in real-time.☆14Aug 1, 2026Updated last month
- XCloud Project's objective is to build similar infrastructure on both AWS and GCP to have multi cloud infrastructure for an organization …☆14Aug 1, 2022Updated 4 years ago
- Get Crypto data from API, stream it to Kafka with Airflow. Write data to MySQL and visualize with Metabase☆17Oct 2, 2023Updated 2 years ago
- ☆18Jul 9, 2019Updated 7 years ago
- Hands-On Tutorial on Building Multimodal RAG Systems☆14Apr 10, 2025Updated last year
- Information system for business project - building and mining data warehouse☆10Jan 11, 2022Updated 4 years ago
- Example of a Streamlit data app powered by Vaex☆11Jul 7, 2022Updated 4 years ago
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- Repo for CDC with debezium blog post☆32Sep 15, 2024Updated last year
- StarCraft 2 Data Pipeline with Airflow, DuckDB and Streamlit☆17Mar 14, 2024Updated 2 years ago
- Tạo cơ sở dữ liệu quản lý hiệu thuốc☆10Dec 4, 2023Updated 2 years ago
- ☆11Oct 8, 2020Updated 5 years ago
- ☆18Aug 26, 2026Updated last week
- ☆14Nov 11, 2023Updated 2 years ago
- This repository contains code and configuration files for an Extract, Transform, Load (ETL) project using Google Cloud Data Fusion for da…☆21Feb 23, 2024Updated 2 years ago
- ☆12Jul 5, 2021Updated 5 years ago
- A dataset for Movie Recommendation with NebulaGraph, ETL to merge two dataset: OMDB & Movielens with dbt, postgres and Nebula-Importer☆17Sep 7, 2024Updated last year
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- An ETL Orchestration using Apache Airflow to extract CSV files from a Google Drive, validate, transform, and load into a PostgreSQL datab…☆28Jun 30, 2024Updated 2 years ago
- how to unit test your PySpark code☆29Mar 26, 2021Updated 5 years ago
- Try to code some algorithms☆17Apr 3, 2025Updated last year
- Formação Engenheiro de Dados Cloud e Big Data (Azure & DataBricks)☆18Oct 31, 2023Updated 2 years ago
- ☆10Feb 27, 2025Updated last year
- ☆13May 19, 2023Updated 3 years ago
- (Bronze medal - 241st place - Top 8%) Repository for the "SIIM-ISIC Melanoma Classification" Kaggle competition.☆13Sep 14, 2020Updated 5 years ago
- ☆14Updated this week
- Telegram бот-планировщик задач с использованием aiogram, celery, Redis, SQLite.☆16Mar 15, 2023Updated 3 years ago
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- Implementation of realworld app backend api☆10Jul 17, 2023Updated 3 years ago
- This project aims to use a convolutional neural network (CNN) to classify 7 classes of skin lesions.☆18Nov 22, 2022Updated 3 years ago
- As skin cancer is one of the most frequent cancers globally, accurate, non-invasive dermoscopy-based diagnosis becomes essential and prom…☆13Aug 2, 2020Updated 6 years ago
- End to end data engineering project with kafka, airflow, spark, postgres and docker.☆118Jan 8, 2026Updated 7 months ago
- Automate process to change image background using Amazon Bedrock and AWS Step Functions☆17Aug 12, 2024Updated 2 years ago
- Hand Dino Game OpenCV Python☆13Feb 19, 2020Updated 6 years ago
- Fivetran's social media reporting dbt package. Combine your Facebook Pages, Instagram Business, Twitter Organic, and LinkedIn Pages socia…☆24Aug 19, 2026Updated 2 weeks ago