Welcome to my data engineering projects repository! Here you will find a collection of data engineering projects that I have worked on.
☆25Apr 27, 2023Updated 3 years ago
Alternatives and similar repositories for data-engineering-projects
Users that are interested in data-engineering-projects are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- A Data Engineering Project that implements an ETL data pipeline using Dagster, Apache Spark, Streamlit, MinIO, Metabase, Dbt, Polars, Doc…☆25Nov 19, 2024Updated last year
- Sample Project to Learn Data Engineering☆10Aug 1, 2021Updated 4 years ago
- Construindo Pipeline de Dados com Astro Python SDK, dbt & Apache Airflow☆10Mar 20, 2024Updated 2 years ago
- Data pipeline to build a data warehouse on Postgres☆15Aug 11, 2024Updated last year
- NoSQL extract, transform, load (ETL) toolkit with Python☆16Updated this week
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- The infomations☆10Dec 13, 2025Updated 7 months ago
- In this project I have built etl pipline which scraps the trending repository based on month,week and day LIVE extract other related info…☆12Sep 9, 2023Updated 2 years ago
- This app is designed to assist applicants in searching for potential jobs and to help recruiters find talented candidates.☆15Feb 16, 2025Updated last year
- Business Intelligence and Data Warehousing Project☆13Dec 4, 2019Updated 6 years ago
- Personal Data Engineering Projects☆1,024Feb 8, 2023Updated 3 years ago
- Spark Structured Streaming data pipeline that processes movie ratings data in real-time.☆14Jul 12, 2026Updated last week
- ☆12Mar 6, 2021Updated 5 years ago
- This is an analytical project done using python to process and extract valuable insights from WhatsApp text file, deployed as a webapp us…☆19Dec 8, 2023Updated 2 years ago
- Hands-On Tutorial on Building Multimodal RAG Systems☆14Apr 10, 2025Updated last year
- 1-Click AI Models by DigitalOcean Gradient • AdDeploy popular AI models on DigitalOcean Gradient GPU virtual machines with just a single click. Zero configuration with optimized deployments.
- Information system for business project - building and mining data warehouse☆10Jan 11, 2022Updated 4 years ago
- Example of a Streamlit data app powered by Vaex☆11Jul 7, 2022Updated 4 years ago
- Integrating Apache Airflow, dbt, Great Expectations and Apache Superset to develop a modern open source data stack.☆20Jun 19, 2022Updated 4 years ago
- Fivetran's Jira source dbt package☆15Oct 1, 2025Updated 9 months ago
- This repository contains code and configuration files for an Extract, Transform, Load (ETL) project using Google Cloud Data Fusion for da…☆20Feb 23, 2024Updated 2 years ago
- ☆21Mar 11, 2025Updated last year
- A dataset for Movie Recommendation with NebulaGraph, ETL to merge two dataset: OMDB & Movielens with dbt, postgres and Nebula-Importer☆17Sep 7, 2024Updated last year
- An ETL Orchestration using Apache Airflow to extract CSV files from a Google Drive, validate, transform, and load into a PostgreSQL datab…☆28Jun 30, 2024Updated 2 years ago
- Try to code some algorithms☆17Apr 3, 2025Updated last year
- Wordpress hosting with auto-scaling - Free Trial Offer • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- Formação Engenheiro de Dados Cloud e Big Data (Azure & DataBricks)☆18Oct 31, 2023Updated 2 years ago
- ☆10Feb 27, 2025Updated last year
- ☆13May 19, 2023Updated 3 years ago
- Designed and implemented a Task Management Application using React, Node.js, Express.js, and MongoDB. Key features include secure JWT aut…☆12Dec 7, 2024Updated last year
- ☆13Updated this week
- velib-v2: An ETL pipeline that employs batch and streaming jobs using Spark, Kafka, Airflow, and other tools, all orchestrated with Docke…☆21Aug 12, 2025Updated 11 months ago
- PDF AI Assistant Powered By Genimi☆12Mar 24, 2024Updated 2 years ago
- ☆15Feb 18, 2022Updated 4 years ago
- As skin cancer is one of the most frequent cancers globally, accurate, non-invasive dermoscopy-based diagnosis becomes essential and prom…☆12Aug 2, 2020Updated 5 years ago
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- End to end data engineering project with kafka, airflow, spark, postgres and docker.☆116Jan 8, 2026Updated 6 months ago
- This project demonstrates how to build and automate an ETL pipeline written in Python and schedule it using open source Apache Airflow or…☆24Aug 21, 2025Updated 11 months ago
- System Design, Solution Architecture, Data Systems Practice☆82Aug 14, 2025Updated 11 months ago
- Automate process to change image background using Amazon Bedrock and AWS Step Functions☆16Aug 12, 2024Updated last year
- Hand Dino Game OpenCV Python☆13Feb 19, 2020Updated 6 years ago
- Exercises & Notes from the LinkedIn Learning courses about Behavioral, Creational & Structural Design Patterns by Bethan Palmer☆18Jun 7, 2025Updated last year
- Labs and demos for courses in the Data Engineer track of GCP Training (http://cloud.google.com/training).☆15Oct 28, 2019Updated 6 years ago