This is a simple ETL using Airflow. First, we fetch data from API (extract). Then, we drop unused columns, convert to CSV, and validate (transform). Finally, we load the transformed data to database (load).
☆24Oct 12, 2019Updated 7 years ago
Alternatives and similar repositories for airflow-etl-learn
Users that are interested in airflow-etl-learn are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Udacity Data Engineering Nano Degree Project, Data Modeling for fact and dimension tables, and ETL pipeline that transfers data from file…☆10Dec 12, 2020Updated 5 years ago
- Training two models, one with with AutoML & one with HyperDrive, compare, and deploy the best model as a service - A Machine Learning Eng…☆16Jan 25, 2022Updated 4 years ago
- ☆10Oct 20, 2022Updated 4 years ago
- Data Vault 2.0: Code generation, Vertica, Airflow☆12Nov 20, 2019Updated 6 years ago
- Handle Big Data for Machine Learning using Python and PySpark, Building ETL Pipelines with PySpark, MongoDB, and Bokeh☆10Nov 12, 2021Updated 4 years ago
- Managed Kubernetes at scale on DigitalOcean • AdDigitalOcean Kubernetes includes the control plane, bandwidth allowance, container registry, automatic updates, and more for free.
- An Airflow pipeline for the collection of historical Twitter data☆10Aug 5, 2019Updated 7 years ago
- AlvinToh Learning Repository for The Ultimate Hands-On Hadoop - Tame your Big Data!☆10May 23, 2018Updated 8 years ago
- This repo contains implementation of various functionalities of various message queues in Python.☆13Aug 13, 2020Updated 6 years ago
- Template to deploy Synapse Analytics using best practices to deliver a proof of concept.☆21Mar 3, 2023Updated 3 years ago
- Data Visualizations for New York City☆11Apr 15, 2020Updated 6 years ago
- Udacity data engineering nanodegree project☆23Nov 8, 2019Updated 6 years ago
- ☆29Aug 10, 2026Updated 2 months ago
- ☆27Mar 7, 2022Updated 4 years ago
- Python Projects with their source code to inspire you .☆21May 28, 2022Updated 4 years ago
- 1-Click AI Models by DigitalOcean Gradient • AdDeploy popular AI models on DigitalOcean Gradient GPU virtual machines with just a single click. Zero configuration with optimized deployments.
- Use fastAPI to generate html web app that will serve a local directory or S3 bucket of images☆11Jan 18, 2021Updated 5 years ago
- Schedule a data pipeline in Google Cloud using cloud function, BigQuery, cloud storage, cloud scheduler, stack trace, cloud build, and p…☆24Jun 4, 2019Updated 7 years ago
- A dashboard is worth a thousand words => https://datastudio.google.com/reporting/755f3183-dd44-4073-804e-9f7d3d993315☆28Oct 30, 2021Updated 5 years ago
- A simple Python Twitter Reply Bot which was made using Tweepy☆17Dec 12, 2022Updated 3 years ago
- A real-time streaming ETL pipeline for streaming and performing sentiment analysis on Twitter data using Apache Kafka, Apache Spark and D…☆29Aug 8, 2020Updated 6 years ago
- A Functional Programming Approach to Composable Bayesian Workflow☆10Aug 1, 2024Updated 2 years ago
- ☆30Jun 12, 2023Updated 3 years ago
- A streamlit app to create a poster from all your Strava activities☆10Jan 16, 2025Updated last year
- Sequential decision-making in JAX.☆10Sep 21, 2026Updated last month
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- This is the source repository for the Programming the Internet of Things - Constrained Device Python code.☆12Oct 2, 2025Updated last year
- Predict performance of a centrifugal chiller using Multiple Linear Regression☆10May 17, 2015Updated 11 years ago
- An introduction to the Observer design patten with C++ using a Weather Station Example☆20Apr 10, 2019Updated 7 years ago
- BFS maze solving program☆14Nov 18, 2018Updated 7 years ago
- Generic decision trees for rust☆12Sep 2, 2018Updated 8 years ago
- ☆13May 12, 2026Updated 5 months ago
- MLflow logging for PyMC☆14Aug 14, 2026Updated 2 months ago
- Variational Auto-Encoder based on Roberta encoder.☆12Oct 31, 2020Updated 6 years ago
- A nice 404 page for pihole with an added automatic script that can be used to whitelist the domain with the click of a button.☆15Nov 2, 2023Updated 3 years ago
- GPU virtual machines on DigitalOcean Gradient AI • AdGet to production fast with high-performance AMD and NVIDIA GPUs you can spin up in seconds. The definition of operational simplicity.
- ☆12Nov 6, 2024Updated 2 years ago
- give me a toml configuration file, I'll give you export MY_ENV=foo☆10Jul 6, 2018Updated 8 years ago
- AI deception research: how LLMs manipulate in competitive games.☆16Mar 8, 2026Updated 7 months ago
- This Github Repository contains everything you need for getting Doom3 running on the raspberry pi. Here you will find the install script …☆11Jan 7, 2022Updated 4 years ago
- Slides and materials for workshop on "Two views on regression with PyMC3 and scikit-learn"☆18Apr 6, 2023Updated 3 years ago
- The constitution for a decentralized autonomous organization for accelerating clinical research through open-source software collaboratio…☆11Apr 21, 2022Updated 4 years ago
- REST API with FastAPI, PostgreSQL, Alembic, SQLAlchemy and Minio (S3 cloud storage)☆13Apr 18, 2023Updated 3 years ago