This is a simple ETL using Airflow. First, we fetch data from API (extract). Then, we drop unused columns, convert to CSV, and validate (transform). Finally, we load the transformed data to database (load).
☆24Oct 12, 2019Updated 6 years ago
Alternatives and similar repositories for airflow-etl-learn
Users that are interested in airflow-etl-learn are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Udacity Data Engineering Nano Degree Project, Data Modeling for fact and dimension tables, and ETL pipeline that transfers data from file…☆10Dec 12, 2020Updated 5 years ago
- Training two models, one with with AutoML & one with HyperDrive, compare, and deploy the best model as a service - A Machine Learning Eng…☆16Jan 25, 2022Updated 4 years ago
- Local SQL Database ---> Azure ---> Power BI☆15Oct 13, 2023Updated 2 years ago
- An Airflow pipeline for the collection of historical Twitter data☆10Aug 5, 2019Updated 7 years ago
- Matching messy Pandas columns with FuzzyWuzzy (Medium Article)☆13Sep 29, 2019Updated 6 years ago
- 1-Click AI Models by DigitalOcean Gradient • AdDeploy popular AI models on DigitalOcean Gradient GPU virtual machines with just a single click. Zero configuration with optimized deployments.
- AlvinToh Learning Repository for The Ultimate Hands-On Hadoop - Tame your Big Data!☆10May 23, 2018Updated 8 years ago
- This is an Analysis to optimise inventory managemnt for FitCapacity company by analyzing sales and inventory data using SQL and PowerBI☆30Sep 6, 2023Updated 2 years ago
- Udacity data engineering nanodegree project☆22Nov 8, 2019Updated 6 years ago
- ☆28Aug 10, 2026Updated last week
- Schedule a data pipeline in Google Cloud using cloud function, BigQuery, cloud storage, cloud scheduler, stack trace, cloud build, and p…☆25Jun 4, 2019Updated 7 years ago
- ☆27Mar 7, 2022Updated 4 years ago
- A dashboard is worth a thousand words => https://datastudio.google.com/reporting/755f3183-dd44-4073-804e-9f7d3d993315☆28Oct 30, 2021Updated 4 years ago
- Data Visualizations for New York City☆11Apr 15, 2020Updated 6 years ago
- A tutorial to setup and deploy a simple Serverless Python workflow with REST API endpoints in AWS Lambda.☆22Apr 22, 2020Updated 6 years ago
- Wordpress hosting with auto-scaling - Free Trial Offer • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- B+ tree implementation in C++ for NTU's CZ4031 course of Database Systems Principles. Supports duplicate keys.☆11Oct 24, 2020Updated 5 years ago
- ☆31Jun 12, 2023Updated 3 years ago
- Active Statistics book web page☆12Jan 3, 2025Updated last year
- The Buisness case of project: Based on given features of dataset we need to predict the performance rating of employee☆17Mar 19, 2025Updated last year
- Spark data pipeline that processes movie ratings data.☆31Aug 1, 2026Updated 2 weeks ago
- A real-time streaming ETL pipeline for streaming and performing sentiment analysis on Twitter data using Apache Kafka, Apache Spark and D…☆29Aug 8, 2020Updated 6 years ago
- Predict performance of a centrifugal chiller using Multiple Linear Regression☆10May 17, 2015Updated 11 years ago
- This project demonstrates how to build and automate an ETL pipeline using DAGs in Airflow and load the transformed data to Bigquery. Ther…☆29Aug 22, 2025Updated 11 months ago
- ☆14May 12, 2026Updated 3 months ago
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- A set of analytics and machine learning models with the goal of bring intelligence to the NFL.☆20May 11, 2020Updated 6 years ago
- SAPProx is a proof of concept tool for intercepting and modifying SAP GUI (DIAG protocol) traffic.☆13Apr 20, 2017Updated 9 years ago
- MLflow logging for PyMC☆14Updated this week
- give me a toml configuration file, I'll give you export MY_ENV=foo☆10Jul 6, 2018Updated 8 years ago
- This repository contains notebooks with different probability density function estimators.☆13Jun 4, 2020Updated 6 years ago
- A Spotify Playlist Recommendation System based on a collaborative filtering algorithm using the Million Playlist Data Set (MPD).☆17Dec 8, 2022Updated 3 years ago
- Python implementation of plot from Kay, Kola, Hullman, Munson "When (ish) is My Bus?" (2016)☆18Dec 19, 2019Updated 6 years ago
- Mastering Gaussian Processes with PyMC☆17Aug 18, 2025Updated last year
- FastAPI Template☆11Jan 11, 2025Updated last year
- Wordpress hosting with auto-scaling - Free Trial Offer • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- C++ programs to transfer a Text file from server to client using TCP☆24Jan 30, 2025Updated last year
- Advent of Code 2025 done in Lean 4☆18Dec 12, 2025Updated 8 months ago
- Projects of CS-537: Intro to Operating Systems (Spring 2019) at University of Wisconsin-Madison using xv6 Operating System☆21May 16, 2019Updated 7 years ago
- The constitution for a decentralized autonomous organization for accelerating clinical research through open-source software collaboratio…☆11Apr 21, 2022Updated 4 years ago
- REST API with FastAPI, PostgreSQL, Alembic, SQLAlchemy and Minio (S3 cloud storage)☆13Apr 18, 2023Updated 3 years ago
- Example repo to create end to end tests for data pipeline.☆25Jun 14, 2024Updated 2 years ago
- Prometheus Exporter for TeslaFi☆14Nov 18, 2023Updated 2 years ago