dogukannulu / aws_end_to_end_streaming_pipeline

Related projects ⓘ

Alternatives and complementary repositories for aws_end_to_end_streaming_pipeline

• dogukannulu / glue_etl_job_data_catalog_s3
  Glue ETL job or EMR Spark that gets from data catalog, modifies and uploads to S3 and Data Catalog
  ☆11Updated last year
• airscholar / realtime-voting-data-engineering
  This repository contains the code for a realtime election voting system. The system is built using Python, Kafka, Spark Streaming, Postgr…
  ☆32Updated 11 months ago
• JesusAcuna / data-engineering-project
  ☆27Updated last year
• HamzaG737 / data-engineering-project
  End to end data engineering project with kafka, airflow, spark, postgres and docker.
  ☆67Updated 3 months ago
• dogukannulu / airflow_kafka_cassandra_mongodb
  Produce Kafka messages, consume them and upload into Cassandra, MongoDB.
  ☆37Updated last year
• afaqueahmad7117 / spark-experiments
  Ultimate guide for mastering Spark Performance Tuning and Optimization concepts and for preparing for Data Engineering interviews
  ☆70Updated 6 months ago
• sidharth1805 / Spotify_etl
  ☆128Updated last year
• josephmachado / python_essentials_for_data_engineers
  Code for blog at https://www.startdataengineering.com/post/python-for-de/
  ☆55Updated 5 months ago
• dogukannulu / csv_extract_airflow_docker
  Writes the CSV file to Postgres, read table and modify it. Write more tables to Postgres with Airflow.
  ☆35Updated last year
• darshilparmar / twitter-airflow-data-engineering-project
  YouTube tutorial project
  ☆94Updated last year
• PacktPublishing / Building-ETL-Pipelines-with-Python
  Building ETL Pipelines with Python
  ☆105Updated 4 months ago
• AhmetFurkanDEMIR / Data-Engineering-Project-with-HDFS-and-Kafka
  Data Engineering Project with Hadoop HDFS and Kafka
  ☆32Updated last year
• airscholar / e2e-data-engineering
  An end-to-end data engineering pipeline that orchestrates data ingestion, processing, and storage using Apache Airflow, Python, Apache Ka…
  ☆204Updated last year
• PacktPublishing / Data-Engineering-with-AWS-2nd-edition
  Data Engineering with AWS, 2nd edition - Published by Packt
  ☆108Updated last year
• SatadruMukherjee / Data-Preprocessing-Models
  ☆60Updated last week
• josephmachado / sde_de101_josephmachado
  Sample repo for startdataengineering DE 101 free course
  ☆35Updated 4 months ago
• itversity / data-engineering-spark
  ☆86Updated 2 years ago
• josephmachado / adv_data_transformation_in_sql
  Code for "Advanced data transformations in SQL" free live workshop
  ☆65Updated 3 weeks ago
• boisalai / de-zoomcamp-2023
  ☆41Updated last year
• PacktPublishing / Data-Engineering-with-Google-Cloud-Platform
  Data Engineering with Google Cloud Platform, published by Packt
  ☆109Updated last year
• vinamrgrover / ETL-Pipeline-Airflow-Reddit-API
  ☆23Updated last year
• priye-1 / airflow_data_pipeline
  ☆16Updated last year
• Kevin-Nduati / My-Spotify-Wrapped
  I will attempt to create my own spotify wrapped by collecting data from the spotify API, perform transformations and create informative d…
  ☆74Updated last year
• mharty3 / energy_data_capstone
  ☆30Updated last year
• ArpiteshSrivastava / spotify-data-engineering-project
  In this project, we will build and ETL(Extract,Transform,Load) pipeline using the Spotify API on AWS. The pipeline will retrieve data fro…
  ☆21Updated last year
• EdidiongEsu / capital_bikeshare
  ☆14Updated 7 months ago
• andrem8 / surf_dash
  ☆132Updated 2 years ago
• itsadityagupta / data-engineering-projects
  Welcome to my data engineering projects repository! Here you will find a collection of data engineering projects that I have worked on.
  ☆15Updated last year
• darshilparmar / dataengineering-youtube-analysis-project
  Data Engineering YouTube Analysis Project by Darshil Parmar
  ☆138Updated 11 months ago
• simardeep1792 / Data-Engineering-Streaming-Project
  ☆38Updated 4 months ago