okzapradhana / etl-flatfile-airflowLinks
Building Data Warehouse on BigQuery which takes flat file as the data sources with Airflow as the Orchestrator
☆12Updated 4 years ago
Alternatives and similar repositories for etl-flatfile-airflow
Users that are interested in etl-flatfile-airflow are comparing it to the libraries listed below
Sorting:
- ☆12Updated last year
- ☆11Updated 3 years ago
- This repository contains an example of how to leverage Cloud Composer and Cloud Dataflow to move data from a Microsoft SQL Server to BigQ…☆19Updated 2 months ago
- How to Automate SQL: dbt(data build tool) tutorial on bigquery with extensive NOTES☆33Updated last year
- Spark, Airflow, Kafka☆26Updated 2 years ago
- Stream Avro SpecificRecord objects in BigQuery using Cloud Dataflow☆13Updated 3 years ago
- learning-by-doing data model built with dbt-core☆14Updated 8 months ago
- Create a streaming data, transfer it to Kafka, modify it with PySpark, take it to ElasticSearch and MinIO☆63Updated 2 years ago
- Data Pipeline from the Global Historical Climatology Network DataSet☆27Updated 2 years ago
- ☆45Updated 2 years ago
- A Series of Notebooks on how to start with Kafka and Python☆152Updated 6 months ago
- Project for "Data pipeline design patterns" blog.☆45Updated last year
- (project & tutorial) dag pipeline tests + ci/cd setup☆88Updated 4 years ago
- A data engineering project with Airflow, dbt, Terrafrom, GCP and much more!☆25Updated 2 years ago
- ☆35Updated 2 years ago
- Near real time ETL to populate a dashboard.☆72Updated last year
- Public source code for the Udemy online course Apache Airflow: Complete Hands-On Beginner to Advanced Class.☆63Updated 4 years ago
- ☆36Updated 8 months ago
- Data lake, data warehouse on GCP☆56Updated 3 years ago
- Final Project of the MLOps Zoomcamp hosted by DataTalksClub.☆26Updated 2 years ago
- Data Engineering with Google Cloud Platform, published by Packt☆118Updated last year
- 🚨 Simple, self-contained fraud detection system built with Apache Kafka and Python☆88Updated 6 years ago
- DataTalks.Club's Data Engineering Zoomcamp Project☆23Updated 3 years ago
- Code for dbt tutorial☆159Updated 2 months ago
- A course by DataTalks Club that covers Spark, Kafka, Docker, Airflow, Terraform, DBT, Big Query etc☆14Updated 3 years ago
- This repository includes end-to-end labs on how to use GCP for applied data science☆14Updated 7 years ago
- End-to-end ELT data engineering project☆22Updated 2 years ago
- A repository of sample code to show data quality checking best practices using Airflow.☆78Updated 2 years ago
- Creation of a data lakehouse and an ELT pipeline to enable the efficient analysis and use of data☆47Updated last year
- Simple ETL pipeline using Python☆27Updated 2 years ago