This is a simple ETL using Airflow. First, we fetch data from API (extract). Then, we drop unused columns, convert to CSV, and validate (transform). Finally, we load the transformed data to database (load).
☆24Oct 12, 2019Updated 6 years ago
Alternatives and similar repositories for airflow-etl-learn
Users that are interested in airflow-etl-learn are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- This project demonstrates how to build and automate an ETL pipeline written in Python and schedule it using open source Apache Airflow or…☆20Aug 21, 2025Updated 7 months ago
- An ETL pipeline that extracts data from S3, stages them in Redshift, and transforms data into a set of dimensional tables☆15May 5, 2020Updated 5 years ago
- ☆15Jan 22, 2017Updated 9 years ago
- A data pipeline moving data from a Relational database system (RDBMS) to a Hadoop file system (HDFS).☆15Jun 3, 2021Updated 4 years ago
- Data Vault 2.0: Code generation, Vertica, Airflow☆13Nov 20, 2019Updated 6 years ago
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting with the flexibility to host WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Cloudways by DigitalOcean.
- 🚚 ETL for Spark and Airflow☆25Mar 19, 2018Updated 8 years ago
- ☆10Oct 20, 2022Updated 3 years ago
- Matching messy Pandas columns with FuzzyWuzzy (Medium Article)☆13Sep 29, 2019Updated 6 years ago
- Data transformation☆23Apr 18, 2021Updated 4 years ago
- A pipeline to CI/CD of a machine learning model on Google Cloud Run☆32May 1, 2023Updated 2 years ago
- An Airflow pipeline for the collection of historical Twitter data☆10Aug 5, 2019Updated 6 years ago
- Schedule a data pipeline in Google Cloud using cloud function, BigQuery, cloud storage, cloud scheduler, stack trace, cloud build, and p…☆26Jun 4, 2019Updated 6 years ago
- AlvinToh Learning Repository for The Ultimate Hands-On Hadoop - Tame your Big Data!☆10May 23, 2018Updated 7 years ago
- A dashboard is worth a thousand words => https://datastudio.google.com/reporting/755f3183-dd44-4073-804e-9f7d3d993315☆28Oct 30, 2021Updated 4 years ago
- DigitalOcean Gradient AI Platform • AdBuild production-ready AI agents using customizable tools or access multiple LLMs through a single endpoint. Create custom knowledge bases or connect external data.
- Active Statistics book web page☆12Jan 3, 2025Updated last year
- ☆32Jun 12, 2023Updated 2 years ago
- Data Visualizations for New York City☆11Apr 15, 2020Updated 5 years ago
- Spark data pipeline that processes movie ratings data.☆31Mar 1, 2026Updated 3 weeks ago
- A simple Python Twitter Reply Bot which was made using Tweepy☆17Dec 12, 2022Updated 3 years ago
- A real-time streaming ETL pipeline for streaming and performing sentiment analysis on Twitter data using Apache Kafka, Apache Spark and D…☆29Aug 8, 2020Updated 5 years ago
- A/B testing - Compare web A and web B☆12Oct 22, 2018Updated 7 years ago
- BFS maze solving program☆14Nov 18, 2018Updated 7 years ago
- Generic decision trees for rust☆12Sep 2, 2018Updated 7 years ago
- GPU virtual machines on DigitalOcean Gradient AI • AdGet to production fast with high-performance AMD and NVIDIA GPUs you can spin up in seconds. The definition of operational simplicity.
- A set of analytics and machine learning models with the goal of bring intelligence to the NFL.☆20May 11, 2020Updated 5 years ago
- My solutions to the projects set in CMU's Intro to Database Systems Course in Rust☆10Jul 14, 2023Updated 2 years ago
- ☆14Mar 18, 2019Updated 7 years ago
- open source smarthome radiator thermostat☆15Jan 3, 2024Updated 2 years ago
- ☆12Nov 6, 2024Updated last year
- This repository contains notebooks with different probability density function estimators.☆14Jun 4, 2020Updated 5 years ago
- Predict performance of a centrifugal chiller using Multiple Linear Regression☆11May 17, 2015Updated 10 years ago
- Helper scripts I use to run many experiments in the morning to check at night☆20Jun 14, 2021Updated 4 years ago
- Python implementation of plot from Kay, Kola, Hullman, Munson "When (ish) is My Bus?" (2016)☆18Dec 19, 2019Updated 6 years ago
- 1-Click AI Models by DigitalOcean Gradient • AdDeploy popular AI models on DigitalOcean Gradient GPU virtual machines with just a single click and start building anything your business needs.
- give me a toml configuration file, I'll give you export MY_ENV=foo☆10Jul 6, 2018Updated 7 years ago
- Timbre Transfer using Differentiable Digital Signal Processing based on DDSP repository☆42Apr 24, 2020Updated 5 years ago
- Mastering Gaussian Processes with PyMC☆18Aug 18, 2025Updated 7 months ago
- C++ programs to transfer a Text file from server to client using TCP☆22Jan 30, 2025Updated last year
- Projects of CS-537: Intro to Operating Systems (Spring 2019) at University of Wisconsin-Madison using xv6 Operating System☆21May 16, 2019Updated 6 years ago
- ☆24Apr 5, 2022Updated 3 years ago
- Demonstrations and visualizations of sorting algorithms (Python and C++).☆22Oct 21, 2018Updated 7 years ago