Banana1206 / TikiAPI-WebScraping
Crawl data from the TIKI e-commerce, designing a data warehouse, implementing an ETL (Extract, Transform, Load) process, and loading the data into MySQL.
☆14Updated last year
Alternatives and similar repositories for TikiAPI-WebScraping:
Users that are interested in TikiAPI-WebScraping are comparing it to the libraries listed below
- Machine Translation using Seq2Seq and Transformer Models☆12Updated last year
- Tiểu Luận Chuyên Ngành☆11Updated 8 months ago
- Nyc_Taxi_Data_Pipeline - DE Project☆104Updated 5 months ago
- Sample code and documentation for very basic things that I can't remember but want to aggregate in one place☆14Updated 3 years ago
- Analyzing Spotify Data with Pyspark and ETL Procedures☆22Updated 6 months ago
- My documents for self-learning fundamental of Data engineering skills☆12Updated last year
- ☆21Updated last year
- Built a real-time streaming pipeline to extract stock data, using Apache Nifi, Debezium, Kafka, and Spark Streaming. Loaded the transform…☆26Updated last year
- For this project I am creating an ETL (Extract, Transform, and Load) pipeline using Python, RegEx, and SQL Database. The goal is to retri…☆27Updated 4 years ago
- In this project I have built etl pipline which scraps the trending repository based on month,week and day LIVE extract other related info…☆12Updated last year
- FInal project for data zoom camp 2024☆18Updated last year
- ☆45Updated 4 years ago
- Writes the CSV file to Postgres, read table and modify it. Write more tables to Postgres with Airflow.☆35Updated last year
- Data Engineering Capstone Project: ETL Pipelines and Data Warehouse Development☆21Updated 5 years ago
- ☆195Updated last year
- ☆28Updated last year
- Public Docker Images for popular services☆26Updated 3 weeks ago
- End-to-end data platform: A PoC Data Platform project utilizing modern data stack (Spark, Airflow, DBT, Trino, Lightdash, Hive metastore,…☆34Updated 5 months ago
- MLOps Implementation for Disaster Tweets Classifier Application☆21Updated last year
- This project aims to predict smartphone prices using a combination of batch and stream processing techniques in a Big Data environment. T…☆11Updated 11 months ago
- In this project, we setup and end to end data engineering using Apache Spark, Azure Databricks, Data Build Tool (DBT) using Azure as our …☆27Updated last year
- Fully dockerized Data Warehouse (DWH) using Airflow, dbt, PostgreSQL and dashboard using redash☆24Updated 2 years ago
- This repository contains the code for a realtime election voting system. The system is built using Python, Kafka, Spark Streaming, Postgr…☆35Updated last year
- This repo gives an introduction to setting up streaming analytics using open source technologies☆24Updated 2 years ago
- This project introduces PySpark, a powerful open-source framework for distributed data processing. We explore its architecture, component…☆27Updated 6 months ago
- Built a Data Pipeline for a Retail store using AWS services that collects data from its transactional database (OLTP) in Snowflake and tr…☆10Updated last year
- Classwork projects and home works done through Udacity data engineering nano degree☆74Updated last year
- Projects done in the Data Engineer Nanodegree Program by Udacity.com☆149Updated 2 years ago
- ☆50Updated last year
- A end-to-end real-time stock market data pipeline with Python, AWS EC2, Apache Kafka, and Cassandra Data is processed on AWS EC2 with Apa…☆25Updated last year