With everything I learned from DEZoomcamp from datatalks.club, this project performs a batch processing on AWS for the cycling dataset which is available on Transport For London (TFL) website. https://cycling.data.tfl.gov.uk
☆15Jan 4, 2026Updated 9 months ago
Alternatives and similar repositories for batch-processing-on-aws
Users that are interested in batch-processing-on-aws are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- ☆36Jun 3, 2023Updated 3 years ago
- A data pipeline with Kafka, Spark Streaming, dbt, Docker, Airflow, and GCP!☆12Jul 6, 2023Updated 3 years ago
- Python or SQL for data transformation☆11Jul 4, 2024Updated 2 years ago
- Processing TfL data for bike usage with Google Cloud Platform.☆46Jul 15, 2022Updated 4 years ago
- Big Data Engineering practice project, including ETL with Airflow and Spark using AWS S3 and EMR☆91Jul 17, 2019Updated 7 years ago
- 1-Click AI Models by DigitalOcean Gradient • AdDeploy popular AI models on DigitalOcean Gradient GPU virtual machines with just a single click. Zero configuration with optimized deployments.
- End to end data engineering project☆59Oct 27, 2022Updated 4 years ago
- The goal of this project is to analyse the impact of Covid-19 on the Aviation industry through data engineering processes using technolog…☆13Jun 26, 2022Updated 4 years ago
- Data processing of OpenSky COVID-19 Flight Dataset✈️☆15Apr 6, 2024Updated 2 years ago
- A course by DataTalks Club that covers Spark, Kafka, Docker, Airflow, Terraform, DBT, Big Query etc☆17Mar 18, 2022Updated 4 years ago
- ☆12Mar 6, 2021Updated 5 years ago
- Toolset for detecting reflected xss in websites☆16Oct 6, 2018Updated 8 years ago
- A simple pipeline utilising cron, Postgres, AWS EC2, and Metabase☆13Jul 9, 2024Updated 2 years ago
- This is a capstone project associated with MLOps Zoomcamp. The end goal of the project is to build an end-to-end machine learning projec…☆14Sep 8, 2022Updated 4 years ago
- ☆12Apr 3, 2026Updated 6 months ago
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- ☆12Jan 14, 2023Updated 3 years ago
- TrafficAdvisor: a Real-Time Traffic Monitoring System☆13Sep 10, 2018Updated 8 years ago
- Fully dockerized Data Warehouse (DWH) using Airflow, dbt, PostgreSQL and dashboard using redash☆26Nov 12, 2022Updated 3 years ago
- real time log event processing using spark, kafka & cassandra☆13Dec 4, 2014Updated 11 years ago
- Semantic subtitle aligner and merger for bilingual subtitle syncing.☆15Oct 10, 2026Updated 3 weeks ago
- An AI-powered system that generates sustainable trip plans and tourism recommendations for Southeast Asia. Built using a Retrieval-Augmen…☆14Oct 8, 2024Updated 2 years ago
- ☆12Oct 24, 2026Updated last week
- MLOps Zoomcamp 2023 repository for Module 3☆11Jun 1, 2023Updated 3 years ago
- This project focuses on generating a diverse and realistic dataset for computer vision training using ChatGPT and a realistic vision imag…☆13Dec 15, 2023Updated 2 years ago
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- Azure Developer CLI extension for local development. One command starts all services with auto-detected dependencies, real-time dashboard…☆12Oct 19, 2026Updated last week
- A demonstration of an ELT (Extract, Load, Transform) pipeline☆32Feb 19, 2024Updated 2 years ago
- ☆30Feb 11, 2024Updated 2 years ago
- Building pipeline to process the real-time data using Spark and Mongodb.☆12Oct 30, 2019Updated 7 years ago
- An API that analyses the housing market real time and presents market opportunities☆28Dec 20, 2022Updated 3 years ago
- Tutorial on NetworkX originally given at NetsciX 2016 School of Code☆15Jul 22, 2024Updated 2 years ago
- This repo contains an azure web application which hosts the azPipelines kubernetes UI to demonstratre that the UI is hostable outside of …☆17Dec 17, 2024Updated last year
- AWS IoT Services sample codes. these will be used in AWS IoT hands-on/workshops in Japan.☆11Jan 14, 2021Updated 5 years ago
- ☆23May 13, 2019Updated 7 years ago
- End-to-end encrypted email - Proton Mail • AdSpecial offer: 40% Off Yearly / 80% Off First Month. All Proton services are open source and independently audited for security.
- ETL flow framework based on Yaml configs in Python☆22Oct 21, 2023Updated 3 years ago
- My final project for the Data Engineering Zoomcamp by DataTalksClub.☆10Apr 6, 2023Updated 3 years ago
- ☆14Oct 1, 2022Updated 4 years ago
- ☆19May 11, 2023Updated 3 years ago
- Fetch real-time NSE stock prices and financial data in Google Sheets with this script. Get market cap, volume, P/E ratio, and more. Just …☆13Feb 6, 2025Updated last year
- ☆16Aug 5, 2023Updated 3 years ago
- This repository contains the code snippets used in "LLM Prompt Engineering For Developers"☆14Apr 22, 2024Updated 2 years ago