mtpatter / time-series-kafka-demoLinks
Fully reproducible, Dockerized, step-by-step, tutorial on how to mock a "real-time" Kafka data stream from a timestamped csv file. Detailed blog post published on Towards Data Science.
☆40Updated 3 years ago
Alternatives and similar repositories for time-series-kafka-demo
Users that are interested in time-series-kafka-demo are comparing it to the libraries listed below
Sorting:
- A Series of Notebooks on how to start with Kafka and Python☆153Updated 5 months ago
- Delta-Lake, ETL, Spark, Airflow☆47Updated 2 years ago
- Full stack data engineering tools and infrastructure set-up☆55Updated 4 years ago
- Code snippets for Data Engineering Design Patterns book☆142Updated 4 months ago
- Apache Airflow Best Practices, published by Packt☆45Updated 9 months ago
- build dw with dbt☆48Updated 9 months ago
- Simple stream processing pipeline☆103Updated last year
- Get data from API, run a scheduled script with Airflow, send data to Kafka and consume with Spark, then write to Cassandra☆141Updated 2 years ago
- Project for real-time anomaly detection using Kafka and python☆58Updated 2 years ago
- A Postgres data warehouse for processing synthetic data using IAC principles☆18Updated 2 years ago
- used Airflow, Postgres, Kafka, Spark, and Cassandra, and GitHub Actions to establish an end-to-end data pipeline☆29Updated last year
- Resources for video demonstrations and blog posts related to DataOps on AWS☆181Updated 3 years ago
- Produce Kafka messages, consume them and upload into Cassandra, MongoDB.☆42Updated last year
- Docker Airflow - Contains a docker compose file for Airflow 2.0☆68Updated 2 years ago
- code snippet for analytics sessions☆34Updated 3 years ago
- Streaming Synthetic Sales Data Generator: Streaming sales data generator for Apache Kafka, written in Python☆44Updated 2 years ago
- Cloned by the `dbt init` task☆61Updated last year
- 📡 Real-time data pipeline with Kafka, Flink, Iceberg, Trino, MinIO, and Superset. Ideal for learning data systems.☆47Updated 6 months ago
- Code for dbt tutorial☆159Updated 2 months ago
- Code for "Advanced data transformations in SQL" free live workshop☆83Updated 3 months ago
- Essential PySpark for Scalable Data Analytics, published by Packt☆45Updated 2 years ago
- This project helps me to understand the core concepts of Apache Airflow. I have created custom operators to perform tasks such as staging…☆92Updated 6 years ago
- Project for "Data pipeline design patterns" blog.☆45Updated last year
- This project shows how to capture changes from postgres database and stream them into kafka☆37Updated last year
- Building a Data Pipeline with an Open Source Stack