End-to-end data platform: A PoC Data Platform project utilizing modern data stack (Spark, Airflow, DBT, Trino, Lightdash, Hive metastore, Minio, Postgres)
☆50Oct 14, 2024Updated 2 years ago
Alternatives and similar repositories for e2e-data-platform
Users that are interested in e2e-data-platform are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Playground for Lakehouse (Iceberg, Hudi, Spark, Flink, Trino, DBT, Airflow, Kafka, Debezium CDC)☆73Sep 23, 2023Updated 3 years ago
- Machine Learning System for Loan Assessment☆15Jul 2, 2026Updated 4 months ago
- On-premises ELT Pipeline☆32Jul 10, 2025Updated last year
- open source data lake☆33Jan 17, 2025Updated last year
- Building a Modern Data Lake with Minio, Spark, Airflow via Docker.☆23May 11, 2024Updated 2 years ago
- Proton VPN Special Offer - Get 70% off • AdSpecial partner offer. Trusted by over 100 million users worldwide. Tested, Approved and Recommended by Experts.
- DWH powered by Clickhouse and dbt☆13Aug 4, 2024Updated 2 years ago
- To provide a deeper understanding of how the modern, open-source data stack consisting of Iceberg, dbt, Trino, and Hive operates within a…☆47Mar 7, 2024Updated 2 years ago
- ☆19Updated this week
- An educational project to build an end-to-end pipline for near real-time and batch processing of data further used for visualisation and …☆10May 19, 2023Updated 3 years ago
- A Data Engineering Project that implements an ETL data pipeline using Dagster, Apache Spark, Streamlit, MinIO, Metabase, Dbt, Polars, Doc…☆25Nov 19, 2024Updated 2 years ago
- GB: Построение хранилища данных и основы ETL☆10Mar 27, 2021Updated 5 years ago
- A project for exploring how Great Expectations can be used to ensure data quality and validate batches within a data pipeline defined in …☆29Aug 30, 2022Updated 4 years ago
- Nyc_Taxi_Data_Pipeline - DE Project☆158Oct 21, 2024Updated 2 years ago
- These are my notes from the book Fundamentals Of Data Engineering.☆20Apr 13, 2025Updated last year
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- An app designed to capture, share, and cherish every unforgettable moment from your special day! Built with Pinata's Files API☆10Oct 13, 2024Updated 2 years ago
- A simple Qt program to easily extract and label samples from videos. Used for dataset creation.☆12Dec 7, 2016Updated 9 years ago
- ☆35Nov 25, 2023Updated 2 years ago
- ☆19Feb 25, 2022Updated 4 years ago
- Large Language Models (LLMs) Learning Resources☆22Jun 16, 2024Updated 2 years ago
- Open Data Stack Platform: a collection of projects and pipelines built with open data stack tools for scalable, observable data platform…☆24Sep 19, 2026Updated 2 months ago
- Nessie Web UI☆12Updated this week
- DataTalks.Club's Data Engineering Zoomcamp Project☆25Jul 14, 2022Updated 4 years ago
- End to End RAG LLM AI Assistant using LangChain, Llama3, Gemma2, OpenAI, FlaskAPI, Grafana☆11Nov 24, 2025Updated 11 months ago
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- Supplementary material for the O'Reilly book "Architecting Data and ML Platforms"☆12Oct 13, 2023Updated 3 years ago
- A minimal implementation of a Litestar application☆18Jul 28, 2026Updated 3 months ago
- Source code for 'Pro Power BI Desktop' by Adam Aspin☆13Mar 28, 2017Updated 9 years ago
- Lightweight reasoning chatbot using LangChain’s modern runnable architecture.☆12Oct 13, 2025Updated last year
- DuckDB with Dashboarding tools demo evidence, streamlit and rill☆23Dec 18, 2023Updated 2 years ago
- An end-to-end AWS data lakehouse leveraging batch data processing, big data technologies, and advanced business analytics.☆11Feb 12, 2026Updated 9 months ago
- ☆10Sep 9, 2020Updated 6 years ago
- ☆13Oct 6, 2024Updated 2 years ago
- Fraud streamhouse with Fluss, Flink and Iceberg☆19Jan 31, 2026Updated 9 months ago
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- Slow & local data allows you to move fast and deliver business value for the 99.9% of the data challenges.☆406Sep 30, 2025Updated last year
- ☆23Feb 8, 2023Updated 3 years ago
- Variable Selection Network with PyTorch☆12May 29, 2024Updated 2 years ago
- ☆14Feb 3, 2020Updated 6 years ago
- My solution for IEEE-CIS Fraud Detection(5th / 6381)☆25Oct 12, 2019Updated 7 years ago
- ☆10Jul 19, 2020Updated 6 years ago
- Data Vault 2.0: Code generation, Vertica, Airflow☆12Nov 20, 2019Updated 7 years ago
Are these results useful?