Welcome to my data engineering projects repository! Here you will find a collection of data engineering projects that I have worked on.
☆25Apr 27, 2023Updated 3 years ago
Alternatives and similar repositories for data-engineering-projects
Users that are interested in data-engineering-projects are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- A Data Engineering Project that implements an ETL data pipeline using Dagster, Apache Spark, Streamlit, MinIO, Metabase, Dbt, Polars, Doc…☆25Nov 19, 2024Updated last year
- Construindo Pipeline de Dados com Astro Python SDK, dbt & Apache Airflow☆10Mar 20, 2024Updated 2 years ago
- Data pipeline to build a data warehouse on Postgres☆15Aug 11, 2024Updated 2 years ago
- The infomations☆10Dec 13, 2025Updated 8 months ago
- In this project I have built etl pipline which scraps the trending repository based on month,week and day LIVE extract other related info…☆12Sep 9, 2023Updated 2 years ago
- Serverless GPU API endpoints on Runpod - Get Bonus Credits • AdSkip the infrastructure headaches. Auto-scaling, pay-as-you-go, no-ops approach lets you focus on innovating your application.
- 🌟 An end-to-end full-stack data science project, including modelling, MLOps, and data storytelling. ✨☆16Aug 30, 2025Updated 11 months ago
- Spark Structured Streaming data pipeline that processes movie ratings data in real-time.☆14Aug 1, 2026Updated last week
- XCloud Project's objective is to build similar infrastructure on both AWS and GCP to have multi cloud infrastructure for an organization …☆14Aug 1, 2022Updated 4 years ago
- ☆12Mar 6, 2021Updated 5 years ago
- ☆18Jul 9, 2019Updated 7 years ago
- Repo for CDC with debezium blog post☆32Sep 15, 2024Updated last year
- StarCraft 2 Data Pipeline with Airflow, DuckDB and Streamlit☆17Mar 14, 2024Updated 2 years ago
- Source Code for 'Azure Data Factory' by Example by Richard Swinbank☆18Jun 21, 2021Updated 5 years ago
- Fivetran's Jira source dbt package☆15Oct 1, 2025Updated 10 months ago
- AI Agents on DigitalOcean Gradient AI Platform • AdBuild production-ready AI agents using customizable tools or access multiple LLMs through a single endpoint. Create custom knowledge bases or connect external data.
- ☆18Updated this week
- ☆12May 22, 2026Updated 2 months ago
- ☆14Nov 11, 2023Updated 2 years ago
- This repository contains code and configuration files for an Extract, Transform, Load (ETL) project using Google Cloud Data Fusion for da…☆20Feb 23, 2024Updated 2 years ago
- ☆20Mar 11, 2025Updated last year
- A dataset for Movie Recommendation with NebulaGraph, ETL to merge two dataset: OMDB & Movielens with dbt, postgres and Nebula-Importer☆17Sep 7, 2024Updated last year
- An ETL Orchestration using Apache Airflow to extract CSV files from a Google Drive, validate, transform, and load into a PostgreSQL datab…☆28Jun 30, 2024Updated 2 years ago
- This extension makes vscode seamlessly work with dbt and bigquery☆15Sep 27, 2022Updated 3 years ago
- Try to code some algorithms☆17Apr 3, 2025Updated last year
- Wordpress hosting with auto-scaling - Free Trial Offer • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- Formação Engenheiro de Dados Cloud e Big Data (Azure & DataBricks)☆18Oct 31, 2023Updated 2 years ago
- ☆10Feb 27, 2025Updated last year
- source code☆17Sep 11, 2025Updated 11 months ago
- ☆13May 19, 2023Updated 3 years ago
- Designed and implemented a Task Management Application using React, Node.js, Express.js, and MongoDB. Key features include secure JWT aut…☆12Dec 7, 2024Updated last year
- Static Fire is an engine for creating static websites, primarily blogs.☆16Jan 22, 2024Updated 2 years ago
- End-to-end ELT data engineering project☆23Dec 24, 2022Updated 3 years ago
- velib-v2: An ETL pipeline that employs batch and streaming jobs using Spark, Kafka, Airflow, and other tools, all orchestrated with Docke…☆21Aug 12, 2025Updated last year
- ☆15Feb 18, 2022Updated 4 years ago
- Managed Database hosting by DigitalOcean • AdPostgreSQL, MySQL, MongoDB, Kafka, Valkey, and OpenSearch available. Automatically scale up storage and focus on building your apps.
- As skin cancer is one of the most frequent cancers globally, accurate, non-invasive dermoscopy-based diagnosis becomes essential and prom…☆13Aug 2, 2020Updated 6 years ago
- End to end data engineering project with kafka, airflow, spark, postgres and docker.☆118Jan 8, 2026Updated 7 months ago
- Part-of-Speech Tagger for Russian language☆23Jul 29, 2020Updated 6 years ago
- Skooldio: Data Pipelines with Airflow☆23May 24, 2025Updated last year
- System Design, Solution Architecture, Data Systems Practice☆82Aug 14, 2025Updated 11 months ago
- This project demonstrates how to build and automate an ETL pipeline written in Python and schedule it using open source Apache Airflow or…☆24Aug 21, 2025Updated 11 months ago
- Exercises & Notes from the LinkedIn Learning courses about Behavioral, Creational & Structural Design Patterns by Bethan Palmer☆18Jun 7, 2025Updated last year