A full data warehouse infrastructure with ETL pipelines running inside docker on Apache Airflow for data orchestration, AWS Redshift for cloud data warehouse and Metabase to serve the needs of data visualizations such as analytical dashboards.
☆138Apr 18, 2020Updated 6 years ago
Alternatives and similar repositories for Skytrax-Data-Warehouse
Users that are interested in Skytrax-Data-Warehouse are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- A way for home buyers to know about factors affecting a state☆48Mar 2, 2019Updated 7 years ago
- Tracking and measuring neighborhood and district-level eviction rates in the city of San Francisco.☆140Jul 14, 2020Updated 6 years ago
- Airflow ETL for Meetup API☆45Dec 27, 2018Updated 7 years ago
- An open source enterprise data warehousing and analysis platform.☆23Nov 8, 2021Updated 4 years ago
- reating a modern data pipeline using a combination of Terraform, AWS Lambda and S3, Snowflake, DBT, Mage AI, and Dash.☆15Jun 26, 2023Updated 3 years ago
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- Data pipeline performing ETL to AWS Redshift using Spark, orchestrated with Apache Airflow☆167Jun 16, 2020Updated 6 years ago
- Udacity Data Engineering Nanodegree Project 3☆11Jul 14, 2019Updated 7 years ago
- RedditR for Content Engagement and Recommendation☆18Dec 21, 2017Updated 8 years ago
- Beginner data engineering project - batch edition☆590Apr 13, 2026Updated 4 months ago
- ☆10May 24, 2021Updated 5 years ago
- Developed a data pipeline to automate data warehouse ETL by building custom airflow operators that handle the extraction, transformation,…☆89Nov 22, 2021Updated 4 years ago
- Use AWS Lambda to Pull E-Scooter and E-Bike Location Data, store in S3 & Redshift using Data Vault Data Model, Server to Google Data Stud…☆16Jun 12, 2022Updated 4 years ago
- 😈Complete End to End ETL Pipeline with Spark, Airflow, & AWS☆51Aug 23, 2019Updated 6 years ago
- This checklist aims to be an exhaustive list of all elements you should consider when using Amazon Redshift.☆15Sep 21, 2020Updated 5 years ago
- Wordpress hosting with auto-scaling - Free Trial Offer • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- Few projects related to Data Engineering including Data Modeling, Infrastructure setup on cloud, Data Warehousing and Data Lake developme…☆1,968Aug 26, 2022Updated 3 years ago
- A collection of data analysis projects done using PySpark via Jupyter notebooks.☆10Oct 8, 2022Updated 3 years ago
- An end-to-end GoodReads Data Pipeline for Building Data Lake, Data Warehouse and Analytics Platform.☆1,540Mar 9, 2020Updated 6 years ago
- An end-to-end data engineering pipeline to create a dashboard for the latest content on the r/Stocks subreddit☆20Aug 5, 2022Updated 4 years ago
- A template for Python projects that need to use a relational database, including tooling for managing schema migrations and testing again…☆13Dec 13, 2024Updated last year
- Example end to end data engineering project.☆1,427Dec 8, 2022Updated 3 years ago
- Example project for consuming AWS Kinesis streamming and save data on Amazon Redshift using Apache Spark☆11May 22, 2018Updated 8 years ago
- Data Engineering pipeline hosted entirely in the AWS ecosystem utilizing DocumentDB as the database☆14Oct 26, 2021Updated 4 years ago
- My Portfolio of all the projects I did for both my Udacity Data Engineer and Data Streaming Nanodegrees☆21Jul 16, 2020Updated 6 years ago
- 1-Click AI Models by DigitalOcean Gradient • AdDeploy popular AI models on DigitalOcean Gradient GPU virtual machines with just a single click. Zero configuration with optimized deployments.
- Runnable e-commerce mini data warehouse based on Python, PostgreSQL & Metabase, template for new projects☆29Mar 31, 2021Updated 5 years ago
- My Insight Data Engineering Fellowship project. I implemented a big data processing pipeline based on lambda architecture, that aggrega…☆509Aug 24, 2022Updated 3 years ago
- Data Engineering Capstone Project: ETL Pipelines and Data Warehouse Development☆22Jul 9, 2019Updated 7 years ago
- pyspark methods to enhance developer productivity 📣 👯 🎉☆687Jun 9, 2026Updated 2 months ago
- Different ways to connect to storage in Azure Databricks☆11Jul 19, 2019Updated 7 years ago
- Spark, Airflow, Kafka☆24Apr 30, 2023Updated 3 years ago
- Personal Data Engineering Projects☆1,027Feb 8, 2023Updated 3 years ago
- ETL (Extract, Transform and Load) with the Spark Python API (PySpark) and Hadoop Distributed File System (HDFS)☆17Dec 18, 2018Updated 7 years ago
- Simple ETL pipeline using Python☆29May 22, 2023Updated 3 years ago
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- Educational project on how to build an ETL (Extract, Transform, Load) data pipeline, orchestrated with Airflow.☆352Jan 12, 2022Updated 4 years ago
- Learning from multiple companies in Silicon Valley. Netflix, Facebook, Google, Startups☆895May 8, 2022Updated 4 years ago
- ☆13Jan 7, 2022Updated 4 years ago
- Data engineering project using UK Bus Open Data Service (BODS) to calculate late buses in real-time for any selected region in England. P…☆32Apr 2, 2023Updated 3 years ago
- Anotações e scripts de web scraping, screen scraping, etc☆18Mar 21, 2018Updated 8 years ago
- Spark Application for analysis of Apache Access logs and detect anamolies! Along with Medium Article.☆21Jan 30, 2019Updated 7 years ago
- Data and analysis related to generational changes in attitudes toward gun control☆10Sep 5, 2022Updated 3 years ago