📡 Real-time data pipeline with Kafka, Flink, Iceberg, Trino, MinIO, and Superset. Ideal for learning data systems.
☆62Jan 18, 2025Updated last year
Alternatives and similar repositories for real-time-data-pipeline
Users that are interested in real-time-data-pipeline are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Local Environment to Practice Data Engineering☆144Dec 30, 2024Updated last year
- Building Data Lakehouse by open source technology. Support end to end data pipeline, from source data on AWS S3 to Lakehouse, visualize a…☆39Dec 15, 2025Updated 3 months ago
- Companion repository that goes along with Snowflake's "Advanced Data Engineering with Snowflake" course☆29Apr 23, 2025Updated 11 months ago
- Repository for Reference for Apache Iceberg LinkedIN Learning Courses☆18Jan 27, 2025Updated last year
- ☆15Apr 18, 2024Updated last year
- DigitalOcean Gradient AI Platform • AdBuild production-ready AI agents using customizable tools or access multiple LLMs through a single endpoint. Create custom knowledge bases or connect external data.
- ☆11Oct 10, 2018Updated 7 years ago
- Open Data Stack Platform: a collection of projects and pipelines built with open data stack tools for scalable, observable data platform…☆22Mar 22, 2026Updated last week
- ☆14Dec 28, 2023Updated 2 years ago
- Data Pipeline that utilizes GCP, Python 3.10, Prefect, and more.☆10Jan 23, 2023Updated 3 years ago
- Files for the Docker and Kubernetes on Google Cloud Hands-On labs☆11Mar 14, 2023Updated 3 years ago
- ☆17May 23, 2025Updated 10 months ago
- ☆22Jul 18, 2024Updated last year
- ☆20Jan 17, 2025Updated last year
- ☆11Nov 26, 2024Updated last year
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting with the flexibility to host WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Cloudways by DigitalOcean.
- ☆11Oct 19, 2023Updated 2 years ago
- ☆12Mar 6, 2021Updated 5 years ago
- Write-Audit-Publish on the lakehouse in pure Python with bauplan and DBOS☆13Jan 8, 2025Updated last year
- This is a simple analytic project using DuckDB & dbt with air quality data.☆24Feb 21, 2024Updated 2 years ago
- Feature demos, integration guides & hands-on labs/projects using Kpow, Flex, Kafka, Flink, Iceberg & more☆50Mar 13, 2026Updated 2 weeks ago
- A Firebase Cloud Function and a Firebase hosted web app to treat weather data collected by Cloud IoT Core☆18Mar 10, 2019Updated 7 years ago
- A tool to learn JSON schema from collection of documents and generate Create table statement for Redshift☆22Jul 12, 2025Updated 8 months ago
- One bite-sized tip or trick for Apache Flink practitioners every day leading up to Christmas Eve 2024.☆29Dec 21, 2024Updated last year
- Local AWS - a lightweight AWS service emulator☆35Mar 22, 2026Updated last week
- Wordpress hosting with auto-scaling on Cloudways • AdFully Managed hosting built for WordPress-powered businesses that need reliable, auto-scalable hosting. Cloudways SafeUpdates now available.
- In this project I have built etl pipline which scraps the trending repository based on month,week and day LIVE extract other related info…☆12Sep 9, 2023Updated 2 years ago
- ☆10Apr 2, 2024Updated last year
- ☆13Mar 30, 2024Updated last year
- ☆10Oct 7, 2024Updated last year
- Streaming Generative AI Application on AWS☆14Jun 24, 2024Updated last year
- 🤖 An autonomous AI agent system that collaboratively designs, implements, and manages Apache Airflow DAGs through natural language inter…☆28Aug 6, 2025Updated 7 months ago
- ☆27Feb 2, 2018Updated 8 years ago
- CI/CD Data Science Project with Rust☆16Sep 27, 2023Updated 2 years ago
- Python FastApi "Circuit Breaker" implementation☆13Mar 14, 2025Updated last year
- Proton VPN Special Offer - Get 70% off • AdSpecial partner offer. Trusted by over 100 million users worldwide. Tested, Approved and Recommended by Experts.
- ☆26Sep 28, 2023Updated 2 years ago
- ☆22Apr 13, 2023Updated 2 years ago
- Let's learn Beam, processing Movie Lens 20m datas. Get top three genres for each user☆14Aug 26, 2018Updated 7 years ago
- Rust server that summarizes text with pre-trained models☆18Mar 8, 2023Updated 3 years ago
- End-to-End deployment of E-commerce customers segmentation using Clustering Machine learning algorithms in Google Cloud Platform and MLOp…☆20Jun 5, 2024Updated last year
- ☆16Oct 18, 2023Updated 2 years ago
- Hourly Electricity Demand Prediction Service☆114Jul 23, 2025Updated 8 months ago