tuanchris / kafka-pipeline
Create a streaming pipeline using Kafka and Kafka Connect
☆14Updated 4 years ago
Related projects ⓘ
Alternatives and complementary repositories for kafka-pipeline
- PyConDE & PyData Berlin 2019 Airflow Workshop: Airflow for machine learning pipelines.☆46Updated last year
- Repo that relates to the Medium blog 'Keeping your ML model in shape with Kafka, Airflow' and MLFlow'☆118Updated last year
- Kafka variant of the MLOps Level 1 stack☆21Updated 2 years ago
- Repo for all my code on the articles I post on medium☆106Updated 2 years ago
- Using Kafka-Python to illustrate a ML production pipeline☆108Updated last year
- This repo is an approach to TDD in machine learning model operation. it covers project structure, testing essentials using pytest with Gi…☆14Updated 3 years ago
- scaffold of Apache Airflow executing Docker containers☆85Updated last year
- A code-based tutorial for production level data streaming with PySpark plus Optimus for data cleaning, Confluent Kafka, & Apache Drill u…☆26Updated 5 years ago
- 🚨 Simple, self-contained fraud detection system built with Apache Kafka and Python☆83Updated 5 years ago
- (project & tutorial) dag pipeline tests + ci/cd setup☆85Updated 3 years ago
- Simple alert system implemented in Kafka and Python☆94Updated 6 years ago
- Ingesting data with Pulumi, AWS lambdas and Snowflake in a scalable, fully replayable manner☆69Updated 2 years ago
- ☆48Updated 3 years ago
- ☆65Updated 4 years ago
- ☆16Updated last year
- PySpark phonetic and string matching algorithms☆35Updated 9 months ago
- ☆19Updated 3 years ago
- Airflow helm chart for AWS EKS☆18Updated 3 years ago
- Bare minimal Airflow on Kubernetes (Local, EKS, AKS)☆52Updated 4 years ago
- Data lake, data warehouse on GCP☆54Updated 2 years ago
- Asynchronous actions for PySpark☆45Updated 2 years ago
- Airflow training for the crunch conf☆105Updated 6 years ago
- Code examples for the Introduction to Kubeflow course☆13Updated 3 years ago
- Set up a 3 node spark cluster using docker containers☆33Updated 6 years ago
- Big Data Demystified meetup and blog examples☆31Updated 3 months ago
- Read Delta tables without any Spark☆47Updated 8 months ago
- Example custom model image trainable and distributable via AWS SageMaker☆36Updated last year
- Code snippets and tools published on the blog at lifearounddata.com☆12Updated 4 years ago