This is a simple ETL using Airflow. First, we fetch data from API (extract). Then, we drop unused columns, convert to CSV, and validate (transform). Finally, we load the transformed data to database (load).
☆24Oct 12, 2019Updated 6 years ago
Alternatives and similar repositories for airflow-etl-learn
Users that are interested in airflow-etl-learn are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Udacity Data Engineering Nano Degree Project, Data Modeling for fact and dimension tables, and ETL pipeline that transfers data from file…☆10Dec 12, 2020Updated 5 years ago
- An ETL pipeline that extracts data from S3, stages them in Redshift, and transforms data into a set of dimensional tables☆16May 5, 2020Updated 6 years ago
- Training two models, one with with AutoML & one with HyperDrive, compare, and deploy the best model as a service - A Machine Learning Eng…☆16Jan 25, 2022Updated 4 years ago
- Local SQL Database ---> Azure ---> Power BI☆15Oct 13, 2023Updated 2 years ago
- ☆15Jan 22, 2017Updated 9 years ago
- Wordpress hosting with auto-scaling - Free Trial Offer • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- A guide to show you how to import data for ETL☆21Dec 8, 2022Updated 3 years ago
- Shunting Yard is a real-time data replication tool that copies data between Hive Metastores.☆20Oct 11, 2021Updated 4 years ago
- Enhanced Fetch API with first class support for JSON objects☆17May 27, 2022Updated 4 years ago
- A data pipeline moving data from a Relational database system (RDBMS) to a Hadoop file system (HDFS).☆15Jun 3, 2021Updated 5 years ago
- Laravel API Boilerplate is a ready-to-use "starting pack" that you can use to build your first API in seconds. As you can easily imagine,…☆10Feb 25, 2017Updated 9 years ago
- 🚚 ETL for Spark and Airflow☆25Mar 19, 2018Updated 8 years ago
- Super simple API for first name to gender conversion.☆12Nov 22, 2017Updated 8 years ago
- Create first API at Reprograma.☆12Dec 11, 2022Updated 3 years ago
- An Airflow pipeline for the collection of historical Twitter data☆10Aug 5, 2019Updated 6 years ago
- AI Agents on DigitalOcean Gradient AI Platform • AdBuild production-ready AI agents using customizable tools or access multiple LLMs through a single endpoint. Create custom knowledge bases or connect external data.
- import os lib = input(""" [1] Download lib & update [2] pass [+] Please Choice >> """) if lib == "1": os.system('pip install reques…☆15Jul 19, 2021Updated 5 years ago
- First-ever code-free and lightweight cross platform API and database test automation tool.☆13Jan 22, 2024Updated 2 years ago
- ⇋ A REST/gRPC server for Instill Model API service☆17May 4, 2026Updated 2 months ago
- Template to deploy Synapse Analytics using best practices to deliver a proof of concept.☆21Mar 3, 2023Updated 3 years ago
- This repo contains implementation of various functionalities of various message queues in Python.☆13Aug 13, 2020Updated 5 years ago
- Developed a multi-user to-do list management web application that gives users the ability to create multiple checklists of tasks, and mar…☆12May 15, 2017Updated 9 years ago
- An open source, privacy-first, self-hosting capable and blazing fast search engine written in JavaScript. Browse anonymously and safely …☆12Sep 10, 2024Updated last year
- Entity Framework Core is a lightweight, extensible, open-sourced version of the Entity Framework Data Access Technology built for .NET Co…☆10May 22, 2020Updated 6 years ago
- NodeJS Clean Architecture. Clean Architecture is an opinionated boilerplate for Node web APIs focused on separation of concerns and scal…☆11Jul 1, 2019Updated 7 years ago
- GPUs on demand by Runpod - Special Offer Available • AdRun AI, ML, and HPC workloads on powerful cloud GPUs—without limits or wasted spend. Deploy GPUs in under a minute and pay by the second.
- ☆10May 13, 2018Updated 8 years ago
- Data transformation☆23Apr 18, 2021Updated 5 years ago
- A microservice-based shopping webapp that uses API-first approach, Spring Boot 3, Kafka 3, Schema Registry, Liquibase, Flyway, gRPC, Grap…☆13Oct 20, 2024Updated last year
- My personal base for React using hooks, typescript, redux, my own api middleware, axios, a personalized webpack config, scss, atomic-desi…☆13Jan 3, 2023Updated 3 years ago
- Laravel API Boilerplate is a ready-to-use "starting pack" that you can use to build your first API in seconds. As you can easily imagine,…☆11Feb 25, 2017Updated 9 years ago
- Use fastAPI to generate html web app that will serve a local directory or S3 bucket of images☆11Jan 18, 2021Updated 5 years ago
- #!/usr/bin/python3 #-*-coding:utf-8-*- # Made With ❤️ By Ahmar And AHMARCODE Project # Update V0.1 # Copyright© Ahmar ID 2021 # 100% Ope…☆15Dec 11, 2021Updated 4 years ago
- ☆27Mar 7, 2022Updated 4 years ago
- A starter repo for Express, using Fern☆16Mar 16, 2026Updated 4 months ago
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- A dashboard is worth a thousand words => https://datastudio.google.com/reporting/755f3183-dd44-4073-804e-9f7d3d993315☆28Oct 30, 2021Updated 4 years ago
- Overview For this assignment you will be developing and implementing : An On-Demand shortest-hop Routing (ODR) protocol for networks of…☆17Apr 10, 2015Updated 11 years ago
- B+ tree implementation in C++ for NTU's CZ4031 course of Database Systems Principles. Supports duplicate keys.☆11Oct 24, 2020Updated 5 years ago
- ADD SUPPORT FOR YOUR COIN The development and services behind Noble were crowdfunded via the sale of the NOXT token. This token can be u…☆19Mar 27, 2017Updated 9 years ago
- We have to build recommender system which recommends tourist travel locations based on his previous rated venues. Recommended engine is b…☆12Jun 11, 2019Updated 7 years ago
- Smart India Hackathon 2018 Identification of meritorious students in primary education Problem Statement:- Gujarat government has near…☆10May 21, 2018Updated 8 years ago
- ☆12Aug 22, 2018Updated 7 years ago