Some example projects for Data Engineers to build, end-to-end.
☆38Nov 8, 2023Updated 2 years ago
Alternatives and similar repositories for DataEngineeringProjects
Users that are interested in DataEngineeringProjects are comparing it to the libraries listed below
Sorting:
- Template for Data Engineering and Data Pipeline projects☆117Jan 1, 2023Updated 3 years ago
- A Data Engineering project. Repository for backend infrastructure and Streamlit app files for a Premier League Dashboard.☆253Dec 19, 2025Updated 3 months ago
- End-to-end data platform leveraging the Modern data stack☆52Apr 10, 2024Updated last year
- how to unit test your PySpark code☆29Mar 26, 2021Updated 4 years ago
- Hexagonal (ports and adapters) architecture applied to Spark and Python data engineering project☆33Jul 26, 2023Updated 2 years ago
- Data engineering project using UK Bus Open Data Service (BODS) to calculate late buses in real-time for any selected region in England. P…☆31Apr 2, 2023Updated 2 years ago
- Add a command to reset the state of any checklists in a document in Obsidian☆21Feb 27, 2026Updated 3 weeks ago
- A complete pipeline to pull data from Scryfall's "Magic: The Gathering"-API, via Prefect orchestration and dbt transformation.☆43Apr 27, 2023Updated 2 years ago
- [under development] ETL materials to support proposal for CDM enhancements for clinical trial data☆24Jun 25, 2021Updated 4 years ago
- ☆21Mar 11, 2025Updated last year
- ☆12Sep 24, 2020Updated 5 years ago
- AWS IoT Services sample codes. these will be used in AWS IoT hands-on/workshops in Japan.☆11Jan 14, 2021Updated 5 years ago
- GitHub Action for use with python package interrogate☆11Nov 12, 2024Updated last year
- A production-grade data pipeline has been designed to automate the parsing of user search patterns to analyze user engagement. Extract d…☆24Nov 22, 2021Updated 4 years ago
- ☆14Jan 3, 2020Updated 6 years ago
- This Guidance helps customers design a resilient batch process application using AWS services☆18Mar 1, 2026Updated 2 weeks ago
- Simplify Big Data Analytics with Amazon EMR, published by Packt☆13Jan 18, 2023Updated 3 years ago
- Using DuckDB with AWS Lambda to process Delta Lake data☆33Jan 26, 2025Updated last year
- A simple script for backing up your favorite YouTube channels.☆12Jan 27, 2024Updated 2 years ago
- This is a custom project for WGU, the original project repo is https://github.com/udacity/nd0821-c2-build-model-workflow-starter☆12Feb 1, 2026Updated last month
- This is a real-life, high throughput streaming ELT data pipeline for ecommerce☆15May 22, 2023Updated 2 years ago
- LLM Building Blocks for Python Course☆17Nov 17, 2025Updated 4 months ago
- ☆15Nov 7, 2024Updated last year
- Scans posts you haven't yet seen in your mastodon timeline and sends the most popular ones to your inbox☆13Jan 5, 2023Updated 3 years ago
- plugin to check spacing between sentences☆10Sep 10, 2023Updated 2 years ago
- ☆19Mar 27, 2020Updated 5 years ago
- ☆16Jul 29, 2021Updated 4 years ago
- Comprehensive Python client for the Uniprot REST API☆56Oct 6, 2025Updated 5 months ago
- This web analytics demo shows how to collect web logs with API Gateway and store them into S3 through Amazon Kinesis. Then this project s…☆21Apr 4, 2025Updated 11 months ago
- An experimental attempt to make a CLI for supply-chain modeling for Helpful Engineering's Project Data☆10Oct 29, 2023Updated 2 years ago
- ☆17Jul 21, 2025Updated 7 months ago
- Courses and projects on Data Camp☆11Jun 28, 2020Updated 5 years ago
- MCP server for Grok AI API integration☆22Jun 2, 2025Updated 9 months ago
- Simple stream processing pipeline☆111Jun 17, 2024Updated last year
- The official Python library for Formulaic☆18Apr 25, 2024Updated last year
- Repository containing projects and summaries of my studies in the field of Data Engineering.☆54Jan 28, 2026Updated last month
- In-browser semantic search demo using EmbeddingGemma and Transformers.js. No server required.☆30Sep 7, 2025Updated 6 months ago
- reating a modern data pipeline using a combination of Terraform, AWS Lambda and S3, Snowflake, DBT, Mage AI, and Dash.☆15Jun 26, 2023Updated 2 years ago
- Project based learning for Data Engineering fundamentals.☆13Jan 15, 2021Updated 5 years ago