WilliamQLiu / python-examples
Simple Python examples including data analysis, ETL, web scraping
☆73Updated last year
Related projects ⓘ
Alternatives and complementary repositories for python-examples
- Code to build a simple analytics data pipeline with Python☆102Updated 7 years ago
- ETL with Python - Taught at DWH course 2017 (TAU)☆101Updated 7 years ago
- Blog post on ETL pipelines with Airflow☆23Updated 4 years ago
- Simple alert system implemented in Kafka and Python☆94Updated 6 years ago
- A guide to show you how to import data for ETL☆20Updated last year
- Demonstration of using Apache Spark to build robust ETL pipelines while taking advantage of open source, general purpose cluster computin…☆24Updated last year
- Course materials for my data pipeline video course with O'Reilly☆194Updated 7 years ago
- Repo for building docker based airflow image. Containers support multiple features like writing logs to local or S3 folder and Initializi…☆32Updated 5 years ago
- 🐍💨 Airflow tutorial for PyCon 2019☆85Updated last year
- Snowflake Cookbook, published by Packt☆73Updated last year
- Data Quest - Data Engineer Learning and Projects☆24Updated 5 years ago
- Airflow workflow management platform chef cookbook.☆68Updated 5 years ago
- ☆21Updated last year
- Slowly Changing Dimension type 2 using Hive query language using exclusive join technique with ORC Hive tables, partitioned and clustered…☆16Updated 5 years ago
- ETL pipeline using pyspark (Spark - Python)☆108Updated 4 years ago
- This repo contains commands that data engineers use in day to day work.☆59Updated last year
- PySpark phonetic and string matching algorithms☆35Updated 9 months ago
- SQL-based transforms compatible with Rasgo and PyRasgo☆24Updated 7 months ago
- Basic tutorial of using Apache Airflow☆35Updated 6 years ago
- Example of an ETL Pipeline using Airflow☆32Updated 7 years ago
- A tutorial on streaming data from a Flask REST API and streaming the response into PostgreSQL☆39Updated 5 years ago
- A repository of sample code to show data quality checking best practices using Airflow.☆72Updated last year
- Using Luigi to create a Machine Learning Pipeline using the Rossman Sales data from Kaggle☆33Updated 8 years ago
- 🚨 Simple, self-contained fraud detection system built with Apache Kafka and Python☆83Updated 5 years ago
- Teaching notes from my Advanced SQL workshops as local lead instructor at General Assembly New York. The first edition was created for th…☆17Updated 4 years ago
- (project & tutorial) dag pipeline tests + ci/cd setup☆85Updated 3 years ago
- ☆16Updated 6 years ago
- Realtime social media data analytics with Apache Spark, Python, Kafka, Pandas, etc☆52Updated 8 years ago
- code, labs and lectures for the course☆45Updated last year