Alternatives and similar repositories for aws-data-pipeline

Users that are interested in aws-data-pipeline are comparing it to the libraries listed below

• hyunjoonbok / PySpark
  PySpark functions and utilities with examples. Assists ETL process of data modeling
  ☆104Updated 4 years ago
• itversity / data-engineering-spark
  ☆88Updated 3 years ago
• supratim94336 / DataEngineeringCapstoneProject
  😈Complete End to End ETL Pipeline with Spark, Airflow, & AWS
  ☆50Updated 6 years ago
• josephmachado / e2e_datapipeline_test
  Example repo to create end to end tests for data pipeline.
  ☆25Updated last year
• damklis / etljob
  Simple ETL pipeline using Python
  ☆28Updated 2 years ago
• alanchn31 / Movalytics-Data-Warehouse
  Data pipeline performing ETL to AWS Redshift using Spark, orchestrated with Apache Airflow
  ☆157Updated 5 years ago
• immu0001 / Udacity-Data-Engineer-nanodegree
  Classwork projects and home works done through Udacity data engineering nano degree
  ☆74Updated last year
• darshilparmar / twitter-airflow-data-engineering-project
  YouTube tutorial project
  ☆105Updated 2 years ago
• JesusAcuna / data-engineering-project
  ☆29Updated 2 years ago
• josephmachado / online_store
  End to end data engineering project
  ☆57Updated 3 years ago
• martandsingh / ApacheSpark
  This repository will help you to learn about databricks concept with the help of examples. It will include all the important topics which…
  ☆103Updated last month
• cartershanklin / pyspark-cheatsheet
  PySpark Cheat Sheet - example code to help you learn PySpark and develop apps faster
  ☆480Updated last year
• josephmachado / data_engineering_best_practices
  Sample project to demonstrate data engineering best practices
  ☆198Updated last year
• mharty3 / energy_data_capstone
  ☆36Updated 2 years ago
• simardeep1792 / Data-Engineering-Streaming-Project
  ☆44Updated last year
• ddgope / Data-Pipelines-with-Airflow
  This project helps me to understand the core concepts of Apache Airflow. I have created custom operators to perform tasks such as staging…
  ☆94Updated 6 years ago
• andrem8 / surf_dash
  ☆161Updated 3 years ago
• josephmachado / adv_data_transformation_in_sql
  Code for "Advanced data transformations in SQL" free live workshop
  ☆87Updated 6 months ago
• rohitrsp898 / Basic_ETL_PySpark
  ☆21Updated 2 years ago
• Joshua-omolewa / Retailstore_ETL_pipeline_project
  Built a Data Pipeline for a Retail store using AWS services that collects data from its transactional database (OLTP) in Snowflake and tr…
  ☆10Updated 2 years ago
• dogukannulu / kafka_spark_structured_streaming
  Get data from API, run a scheduled script with Airflow, send data to Kafka and consume with Spark, then write to Cassandra
  ☆143Updated 2 years ago
• ajupton / big-data-engineering-project
  Big Data Engineering practice project, including ETL with Airflow and Spark using AWS S3 and EMR
  ☆88Updated 6 years ago
• sidharth1805 / Spotify_etl
  ☆142Updated 2 years ago
• afaqueahmad7117 / spark-experiments
  Ultimate guide for mastering Spark Performance Tuning and Optimization concepts and for preparing for Data Engineering interviews
  ☆173Updated 2 months ago
• AuFeld / Data_Engineering_Projects
  A collection of data engineering projects: data modeling, ETL pipelines, data lakes, infrastructure configuration on AWS, data warehousin…
  ☆15Updated 4 years ago
• HamzaG737 / data-engineering-project
  End to end data engineering project with kafka, airflow, spark, postgres and docker.
  ☆105Updated 7 months ago
• SatadruMukherjee / Data-Preprocessing-Models
  ☆70Updated last week
• vim89 / datapipelines-essentials-python
  Simplified ETL process in Hadoop using Apache Spark. Has complete ETL pipeline for datalake. SparkSession extensions, DataFrame validatio…
  ☆55Updated 2 years ago
• arverma / TowardsDataEngineering
  This repo contains commands that data engineers use in day to day work.
  ☆61Updated 2 years ago
• Modingwa / Data-Engineering-Capstone-Project
  Udacity Data Engineering Nanodegree Capstone Project
  ☆36Updated 5 years ago