This is a simple ETL using Airflow. First, we fetch data from API (extract). Then, we drop unused columns, convert to CSV, and validate (transform). Finally, we load the transformed data to database (load).
☆24Oct 12, 2019Updated 7 years ago
Alternatives and similar repositories for airflow-etl-learn
Users that are interested in airflow-etl-learn are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- An ETL pipeline that extracts data from S3, stages them in Redshift, and transforms data into a set of dimensional tables☆16May 5, 2020Updated 6 years ago
- Local SQL Database ---> Azure ---> Power BI☆15Oct 13, 2023Updated 3 years ago
- ☆15Jan 22, 2017Updated 9 years ago
- A data pipeline moving data from a Relational database system (RDBMS) to a Hadoop file system (HDFS).☆15Jun 3, 2021Updated 5 years ago
- This solution presents a traditional 3-tier architecture in AWS.☆15May 1, 2023Updated 3 years ago
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- Data Vault 2.0: Code generation, Vertica, Airflow☆12Nov 20, 2019Updated 6 years ago
- 🚚 ETL for Spark and Airflow☆25Mar 19, 2018Updated 8 years ago
- ☆10Oct 20, 2022Updated 3 years ago
- An Airflow pipeline for the collection of historical Twitter data☆10Aug 5, 2019Updated 7 years ago
- AlvinToh Learning Repository for The Ultimate Hands-On Hadoop - Tame your Big Data!☆10May 23, 2018Updated 8 years ago
- Template to deploy Synapse Analytics using best practices to deliver a proof of concept.☆21Mar 3, 2023Updated 3 years ago
- ☆29Aug 10, 2026Updated 2 months ago
- A tutorial on how to run Apache Spark on Docker☆29Sep 28, 2020Updated 6 years ago
- A tutorial to setup and deploy a simple Serverless Python workflow with REST API endpoints in AWS Lambda.☆22Apr 22, 2020Updated 6 years ago
- Wordpress hosting with auto-scaling - Free Trial Offer • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- B+ tree implementation in C++ for NTU's CZ4031 course of Database Systems Principles. Supports duplicate keys.☆11Oct 24, 2020Updated 5 years ago
- ☆12Aug 22, 2018Updated 8 years ago
- Active Statistics book web page☆12Jan 3, 2025Updated last year
- Spark data pipeline that processes movie ratings data.☆31Oct 1, 2026Updated 2 weeks ago
- Predict performance of a centrifugal chiller using Multiple Linear Regression☆10May 17, 2015Updated 11 years ago
- A simple Python Twitter Reply Bot which was made using Tweepy☆17Dec 12, 2022Updated 3 years ago
- A/B testing - Compare web A and web B☆12Oct 22, 2018Updated 7 years ago
- This project demonstrates how to build and automate an ETL pipeline using DAGs in Airflow and load the transformed data to Bigquery. Ther…☆28Aug 22, 2025Updated last year
- ☆13May 12, 2026Updated 5 months ago
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- An introduction to the Observer design patten with C++ using a Weather Station Example☆20Apr 10, 2019Updated 7 years ago
- A set of analytics and machine learning models with the goal of bring intelligence to the NFL.☆20May 11, 2020Updated 6 years ago
- Jira shell scripts☆39Nov 18, 2023Updated 2 years ago
- SAPProx is a proof of concept tool for intercepting and modifying SAP GUI (DIAG protocol) traffic.☆13Apr 20, 2017Updated 9 years ago
- BFS maze solving program☆14Nov 18, 2018Updated 7 years ago
- Variational Auto-Encoder based on Roberta encoder.☆12Oct 31, 2020Updated 5 years ago
- MLflow logging for PyMC☆14Aug 14, 2026Updated 2 months ago
- This repository contains notebooks with different probability density function estimators.☆13Jun 4, 2020Updated 6 years ago
- Helper scripts I use to run many experiments in the morning to check at night☆20Jun 14, 2021Updated 5 years ago
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- Mastering Gaussian Processes with PyMC☆18Aug 18, 2025Updated last year
- FastAPI Template☆11Jan 11, 2025Updated last year
- C++ programs to transfer a Text file from server to client using TCP☆24Jan 30, 2025Updated last year
- Advent of Code 2025 done in Lean 4☆18Dec 12, 2025Updated 10 months ago
- Projects of CS-537: Intro to Operating Systems (Spring 2019) at University of Wisconsin-Madison using xv6 Operating System☆22May 16, 2019Updated 7 years ago
- AI deception research: how LLMs manipulate in competitive games.☆16Mar 8, 2026Updated 7 months ago
- An example application to implement unit and integration tests using test database☆19Mar 28, 2022Updated 4 years ago