Banana1206 / TikiAPI-WebScraping
Crawl data from the TIKI e-commerce, designing a data warehouse, implementing an ETL (Extract, Transform, Load) process, and loading the data into MySQL.
☆14Updated last year
Alternatives and similar repositories for TikiAPI-WebScraping:
Users that are interested in TikiAPI-WebScraping are comparing it to the libraries listed below
- Machine Translation using Seq2Seq and Transformer Models☆12Updated last year
- Tiểu Luận Chuyên Ngành☆12Updated 9 months ago
- Nyc_Taxi_Data_Pipeline - DE Project☆105Updated 6 months ago
- Resources and projects from Udacity Data Engineering with AWS nano degree programme☆25Updated 2 years ago
- Sample code and documentation for very basic things that I can't remember but want to aggregate in one place☆14Updated 3 years ago
- Writes the CSV file to Postgres, read table and modify it. Write more tables to Postgres with Airflow.☆35Updated last year
- Built a real-time streaming pipeline to extract stock data, using Apache Nifi, Debezium, Kafka, and Spark Streaming. Loaded the transform…☆26Updated last year
- In this project, we will build and ETL(Extract,Transform,Load) pipeline using the Spotify API on AWS. The pipeline will retrieve data fro…☆21Updated last year
- End to end data engineering project with kafka, airflow, spark, postgres and docker.☆91Updated last month
- Built a Data Pipeline for a Retail store using AWS services that collects data from its transactional database (OLTP) in Snowflake and tr…☆9Updated last year
- My documents for self-learning fundamental of Data engineering skills☆12Updated last year
- ☆58Updated 8 months ago
- Analyzing Spotify Data with Pyspark and ETL Procedures☆22Updated 6 months ago
- This project aims to predict smartphone prices using a combination of batch and stream processing techniques in a Big Data environment. T…☆12Updated last year
- An end-to-end data engineering pipeline that orchestrates data ingestion, processing, and storage using Apache Airflow, Python, Apache Ka…☆244Updated 2 months ago
- Welcome to my data engineering projects repository! Here you will find a collection of data engineering projects that I have worked on.☆17Updated 2 years ago
- ☆21Updated last year
- ☆29Updated last year
- ☆40Updated 9 months ago
- Price Crawler - Tracking Price Inflation☆185Updated 4 years ago
- Classwork projects and home works done through Udacity data engineering nano degree☆74Updated last year
- A custom end-to-end analytics platform for customer churn☆11Updated 3 months ago
- A Data Engineering Project that implements an ETL data pipeline using Dagster, Apache Spark, Streamlit, MinIO, Metabase, Dbt, Polars, Doc…☆21Updated 5 months ago
- velib-v2: An ETL pipeline that employs batch and streaming jobs using Spark, Kafka, Airflow, and other tools, all orchestrated with Docke…☆18Updated 7 months ago
- DataTalks.Club's Data Engineering Zoomcamp Project☆23Updated 2 years ago
- Produce Kafka messages, consume them and upload into Cassandra, MongoDB.☆41Updated last year
- ☆27Updated last year
- Get data from API, run a scheduled script with Airflow, send data to Kafka and consume with Spark, then write to Cassandra☆137Updated last year
- I am using confluent Kafka cluster to produce and consume scraped data. In this project, I've created a real-time data pipeline that uti…☆29Updated last year
- Spark all the ETL Pipelines☆32Updated last year