hellokoding / kafka-connect-sink-postgres-with-avro-schemaregistry-python
Streaming Data from Kafka to Postgres with Kafka Connect, AVRO, Schema Registry and Python
☆14Updated 6 years ago
Related projects ⓘ
Alternatives and complementary repositories for kafka-connect-sink-postgres-with-avro-schemaregistry-python
- Fully reproducible, Dockerized, step-by-step, demo on how to stream tables from Postgres to Kafka/KSQL back to Postgres. Detailed blog p…☆148Updated 2 years ago
- Docker compose files for various kafka stacks☆33Updated 6 years ago
- Real-time anomaly detection using Kafka, KSQL User Defined Function and a pre-trained model☆30Updated 10 months ago
- Kafka Connect Examples☆42Updated 2 years ago
- event-triggered plugins for airflow☆21Updated 4 years ago
- Common components used across the datamountaineer kafka connect connectors☆21Updated 3 years ago
- Realtime social media data analytics with Apache Spark, Python, Kafka, Pandas, etc☆52Updated 8 years ago
- AWS Lambda function to get events in Kafka topic when files are uploaded to S3☆24Updated 6 years ago
- A toolset to streamline running spark python on EMR☆20Updated 7 years ago
- Various Demos mostly based on docker environments☆33Updated last year
- 🚨 Simple, self-contained fraud detection system built with Apache Kafka and Python☆83Updated 5 years ago
- Stream Processing Workshop☆20Updated 3 months ago
- Kafka Connect Sink/Source for Twitter☆90Updated 4 years ago
- Simple alert system implemented in Kafka and Python☆94Updated 6 years ago
- This repo demonstrates how to load a sample Parquet formatted file from an AWS S3 Bucket. A python job will then be submitted to a Apach…☆19Updated 8 years ago
- ☆25Updated 5 years ago
- Solving data streaming problems using joins, processor, punctuator and state store☆11Updated 3 years ago
- Big Data Demystified meetup and blog examples☆31Updated 2 months ago
- Supporting repository for the blog post at https://medium.com/@stephane.maarek/how-to-use-apache-kafka-to-transform-a-batch-pipeline-into…☆241Updated 10 months ago
- ☆24Updated 8 years ago
- Twitter sentiment analysis using Spark and Stanford CoreNLP and visualization using elasticsearch and kibana☆20Updated 6 years ago
- Spark with Scala example projects☆33Updated 5 years ago
- ☆23Updated 5 years ago
- Ingest tweets with Kafka. Use Spark to track popular hashtags and trendsetters for each hashtag☆29Updated 8 years ago
- Real-time report dashboard with Apache Kafka, Apache Spark Streaming and Node.js☆49Updated last year
- Basic tutorial of using Apache Airflow☆35Updated 6 years ago
- Simplified ETL process in Hadoop using Apache Spark. Has complete ETL pipeline for datalake. SparkSession extensions, DataFrame validatio…☆53Updated last year
- This project describes how to write full ETL data pipeline using spark.☆15Updated 2 years ago
- Use Kafka and Apache Spark streaming to perform click stream analytics☆76Updated 4 years ago
- Docker container for Kafka - Spark Streaming - Cassandra☆97Updated 5 years ago