erkansirin78 / data-generator
This repo is for generating data from existing dataset to a file or producing dataset rows as message to kafka in a streaming manner.
☆21Updated 5 months ago
Related projects ⓘ
Alternatives and complementary repositories for data-generator
- This is a simple iris flower classification model deployment project as flask app on Docker or Kubernetes.☆11Updated 2 years ago
- This repo contains datasets used in trainings.☆49Updated 3 weeks ago
- Bu repo 3-5 gün süreyle düzenlenen Python ile Makine Öğrenmesi Eğitimleri için oluşturulmuştur.☆20Updated 4 years ago
- Create a streaming data, transfer it to Kafka, modify it with PySpark, take it to ElasticSearch and MinIO☆57Updated last year
- ☆38Updated 4 months ago
- Glue ETL job or EMR Spark that gets from data catalog, modifies and uploads to S3 and Data Catalog☆11Updated last year
- Get data from API, run a scheduled script with Airflow, send data to Kafka and consume with Spark, then write to Cassandra☆128Updated last year
- ☆86Updated 2 years ago
- PySpark functions and utilities with examples. Assists ETL process of data modeling☆99Updated 3 years ago
- ☆14Updated last year
- ☆27Updated last year
- An End-to-End ETL data pipeline that leverages pyspark parallel processing to process about 25 million rows of data coming from a SaaS ap…☆26Updated last year
- ☆30Updated last year
- Writes the CSV file to Postgres, read table and modify it. Write more tables to Postgres with Airflow.☆35Updated last year
- Series follows learning from Apache Spark (PySpark) with quick tips and workaround for daily problems in hand☆42Updated last year
- PySpark Cheat Sheet - example code to help you learn PySpark and develop apps faster☆424Updated last month
- ☆23Updated last year
- A course by DataTalks Club that covers Spark, Kafka, Docker, Airflow, Terraform, DBT, Big Query etc☆11Updated 2 years ago
- Classwork projects and home works done through Udacity data engineering nano degree☆74Updated 11 months ago
- ☆41Updated last year
- Course Material Data Engineering on AWS Course☆28Updated 2 months ago
- Solution to all projects of Udacity's Data Engineering Nanodegree: Data Modeling with Postgres & Cassandra, Data Warehouse with Redshift,…☆56Updated 2 years ago
- Courses and projects on Data Camp☆11Updated 4 years ago
- Data Engineering Project with Hadoop HDFS and Kafka☆32Updated last year
- End-to-end ELT data engineering project☆20Updated last year
- Data Engineering with Google Cloud Platform, published by Packt☆109Updated last year
- ☆113Updated last month
- Projects done in the Data Engineer Nanodegree Program by Udacity.com☆94Updated last year