mtpatter / time-series-kafka-demoLinks
Fully reproducible, Dockerized, step-by-step, tutorial on how to mock a "real-time" Kafka data stream from a timestamped csv file. Detailed blog post published on Towards Data Science.
☆41Updated 3 years ago
Alternatives and similar repositories for time-series-kafka-demo
Users that are interested in time-series-kafka-demo are comparing it to the libraries listed below
Sorting:
- A Series of Notebooks on how to start with Kafka and Python☆152Updated 7 months ago
- used Airflow, Postgres, Kafka, Spark, and Cassandra, and GitHub Actions to establish an end-to-end data pipeline☆29Updated last year
- Get data from API, run a scheduled script with Airflow, send data to Kafka and consume with Spark, then write to Cassandra☆143Updated 2 years ago
- Simple stream processing pipeline☆109Updated last year
- Data Engineering examples for Airflow, Prefect; dbt for BigQuery, Redshift, ClickHouse, Postgres, DuckDB; PySpark for Batch processing; K…☆68Updated 3 months ago
- The Python fake data producer for Apache Kafka® is a complete demo app allowing you to quickly produce JSON fake streaming datasets and …☆85Updated last year
- Code for dbt tutorial☆161Updated 3 weeks ago
- This project focuses on building a robust data pipeline using Apache Airflow to automate the ingestion of weather data from the OpenWeath…☆22Updated 2 years ago
- Some recipes for data engineering with Python☆23Updated 4 years ago
- ☆88Updated 3 years ago
- Source Code for our Simple Walkthrough Videos☆70Updated last year
- Data Engineering with AWS, 2nd edition - Published by Packt☆155Updated last year
- Project for "Data pipeline design patterns" blog.☆46Updated last year
- Code snippets for Data Engineering Design Patterns book☆207Updated 6 months ago
- A list of all my posts and personal projects☆74Updated last year
- Dockerizing an Apache Spark Standalone Cluster☆43Updated 3 years ago
- build dw with dbt☆46Updated 11 months ago
- ☆44Updated last year
- A course by DataTalks Club that covers Spark, Kafka, Docker, Airflow, Terraform, DBT, Big Query etc☆14Updated 3 years ago
- This project shows how to capture changes from postgres database and stream them into kafka☆38Updated last year
- Full stack data engineering tools and infrastructure set-up☆56Updated 4 years ago
- Duke MIDS: Data Engineering and DataOps Course☆67Updated 8 months ago
- Repo for Climate AI Hackathon☆24Updated 2 years ago
- Delta-Lake, ETL, Spark, Airflow☆48Updated 2 years ago
- A Postgres data warehouse for processing synthetic data using IAC principles☆19Updated 2 years ago
- Apache Flink (Pyflink) and Related Projects☆41Updated 5 months ago
- Simple ETL pipeline using Python☆28Updated 2 years ago
- Resources for video demonstrations and blog posts related to DataOps on AWS☆182Updated 3 years ago
- This repository contains the code for a realtime election voting system. The system is built using Python, Kafka, Spark Streaming, Postgr…☆41Updated last year
- Data engineering project using UK Bus Open Data Service (BODS) to calculate late buses in real-time for any selected region in England. P…☆30Updated 2 years ago