π‘ Real-time data pipeline with Kafka, Flink, Iceberg, Trino, MinIO, and Superset. Ideal for learning data systems.
β71Jan 18, 2025Updated last year
Alternatives and similar repositories for real-time-data-pipeline
Users that are interested in real-time-data-pipeline are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Local Environment to Practice Data Engineeringβ144Dec 30, 2024Updated last year
- β22Dec 18, 2024Updated last year
- Building Data Lakehouse by open source technology. Support end to end data pipeline, from source data on AWS S3 to Lakehouse, visualize aβ¦β40Dec 15, 2025Updated 4 months ago
- Resources to development of data analytics pathβ24Apr 20, 2026Updated 2 weeks ago
- Repository for Reference for Apache Iceberg LinkedIN Learning Coursesβ18Jan 27, 2025Updated last year
- Virtual machines for every use case on DigitalOcean β’ AdGet dependable uptime with 99.99% SLA, simple security tools, and predictable monthly pricing with DigitalOcean's virtual machines, called Droplets.
- A practical demo using Atomic Agents, showing how to build your own code generation agent that actually executes the code it writes in a β¦β15Nov 24, 2024Updated last year
- Templates to get started with Timescale on Finance or Sensors (IoT)β21Apr 30, 2025Updated last year
- Fast data quality framework for modern data infrastructureβ29Apr 2, 2026Updated last month
- β13Dec 28, 2023Updated 2 years ago
- Open-source, cost-effective end-to-end logging system featuring full architecture, deployment automation, dashboards, and production-readβ¦β88Jan 27, 2026Updated 3 months ago
- Build, automate, and scale your entire company with a digital workforce. JeweledTech's Agentic OS is an open-source framework for scaffolβ¦β15Feb 23, 2026Updated 2 months ago
- Files for the Docker and Kubernetes on Google Cloud Hands-On labsβ11Mar 14, 2023Updated 3 years ago
- Azure DP 900 notesβ10Jan 23, 2022Updated 4 years ago
- KnetBuilder data integration platform for building knowledge graphs. Previously known as ondex.β15Apr 2, 2026Updated last month
- Deploy to Railway using AI coding agents - Free Credits Offer β’ AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- β11Oct 19, 2023Updated 2 years ago
- This is a simple analytic project using DuckDB & dbt with air quality data.β24Feb 21, 2024Updated 2 years ago
- β12Mar 6, 2021Updated 5 years ago
- Write-Audit-Publish on the lakehouse in pure Python with bauplan and DBOSβ13Jan 8, 2025Updated last year
- β37Feb 9, 2026Updated 3 months ago
- β10Nov 2, 2023Updated 2 years ago
- A Firebase Cloud Function and a Firebase hosted web app to treat weather data collected by Cloud IoT Coreβ18Mar 10, 2019Updated 7 years ago
- trino monitoring with JMX metrics through Prometheus and Grafanaβ17Aug 14, 2024Updated last year
- A tool to learn JSON schema from collection of documents and generate Create table statement for Redshiftβ22Jul 12, 2025Updated 9 months ago
- Bare Metal GPUs on DigitalOcean Gradient AI β’ AdPurpose-built for serious AI teams training foundational models, running large-scale inference, and pushing the boundaries of what's possible.
- One bite-sized tip or trick for Apache Flink practitioners every day leading up to Christmas Eve 2024.β29Dec 21, 2024Updated last year
- Local AWS - a lightweight AWS service emulatorβ43Apr 26, 2026Updated 2 weeks ago
- Short Range Ultrasonic Radar - A simple radar using the ultrasonic sensor, this radar works by measuring a range from 3cm to 40 cm as nonβ¦β19Nov 11, 2024Updated last year
- Dynamic tiles from CMR queriesβ25May 1, 2026Updated last week
- β10Apr 2, 2024Updated 2 years ago
- In this project I have built etl pipline which scraps the trending repository based on month,week and day LIVE extract other related infoβ¦β12Sep 9, 2023Updated 2 years ago
- A Python CLI application that demonstrates how you can access AWS services, such as Amazon S3 and Amazon Athena, using trusted identity pβ¦β13Mar 11, 2025Updated last year
- β16Apr 8, 2024Updated 2 years ago
- β26Jul 30, 2024Updated last year
- GPU virtual machines on DigitalOcean Gradient AI β’ AdGet to production fast with high-performance AMD and NVIDIA GPUs you can spin up in seconds. The definition of operational simplicity.
- β13Mar 30, 2024Updated 2 years ago
- Streaming Generative AI Application on AWSβ14Jun 24, 2024Updated last year
- π€ An autonomous AI agent system that collaboratively designs, implements, and manages Apache Airflow DAGs through natural language interβ¦β28Aug 6, 2025Updated 9 months ago
- Automate data collection from Spotify's worldwide ranking in 50+ countriesβ25May 3, 2020Updated 6 years ago
- CI/CD Data Science Project with Rustβ16Sep 27, 2023Updated 2 years ago
- β26Sep 28, 2023Updated 2 years ago
- Python FastApi "Circuit Breaker" implementationβ13Mar 14, 2025Updated last year