The goal of this project is to offer an AWS EMR template using Spot Fleet and On-Demand Instances that you can use quickly. Just focus on writing pyspark code.
☆28Jun 13, 2022Updated 4 years ago
Alternatives and similar repositories for pyspark-on-aws-emr
Users that are interested in pyspark-on-aws-emr are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Scheduling Big Data Workloads and Data Pipelines in the Cloud with pyDag☆23Sep 19, 2022Updated 4 years ago
- Challenge Data Engineer☆25Jun 13, 2022Updated 4 years ago
- Dockerizing and Consuming an Apache Livy environment☆13Jun 29, 2022Updated 4 years ago
- Dockerizing a Python Script for Web Scraping and consume the scraped data using FastApi (www.metroscubicos.com)☆15Dec 16, 2021Updated 4 years ago
- An end-to-end data engineering pipeline to create a dashboard for the latest content on the r/Stocks subreddit☆20Aug 5, 2022Updated 4 years ago
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- PySpark Projects☆28Oct 22, 2026Updated last week
- This repository provides Code examples written in Python,Spark-Scala using primarily boto3 SDK API methods and aws cli examples for major…☆14Mar 6, 2022Updated 4 years ago
- Build and run Spark Structured Streaming pipelines in Hadoop - project using PySpark.☆13Jun 6, 2019Updated 7 years ago
- My applied big data analytic project with pyspark.☆10Sep 21, 2022Updated 4 years ago
- Local Development of AWS Glue with Docker and Visual Studio Code☆14Nov 29, 2021Updated 4 years ago
- End to End Mlops System to predict the air quality in Cameroon. Everything is automatic from the Data pipeline, to model retraining and r…☆11Dec 6, 2024Updated last year
- Creating This Repo For Various Spark Examples☆12Feb 13, 2026Updated 8 months ago
- A collection of data analysis projects done using PySpark via Jupyter notebooks.☆10Oct 8, 2022Updated 4 years ago
- Collection Of All Python Notebooks Shared On Twitter☆11Apr 10, 2023Updated 3 years ago
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- ecommerce GCP Streaming pipeline ― Cloud Storage, Compute Engine, Pub/Sub, Dataflow, Apache Beam, BigQuery and Tableau; GCP Batch pipelin…☆10Mar 9, 2022Updated 4 years ago
- The Free AWS Certified Cloud Practitioner Study Course☆18Oct 15, 2019Updated 7 years ago
- aigc evals☆10Dec 2, 2023Updated 2 years ago
- ☆10Jun 29, 2023Updated 3 years ago
- A project for the development of rich geospatial data from the city of São Paulo for use in Machine Learning models.☆12Jul 4, 2021Updated 5 years ago
- A fast and low memory requirement version of PointHop and PointHop++, which is built upon Apache Spark.☆10Jul 14, 2020Updated 6 years ago
- This is the first project where we worked on apache spark, In this project what we have done is that we downloaded the datasets from KAGG…☆23Oct 14, 2021Updated 5 years ago
- Code snippets and tools published on the blog at lifearounddata.com☆12Jan 19, 2020Updated 6 years ago
- Spark Projects for the Berkeley Data Science Course☆13Aug 12, 2015Updated 11 years ago
- Managed Kubernetes at scale on DigitalOcean • AdDigitalOcean Kubernetes includes the control plane, bandwidth allowance, container registry, automatic updates, and more for free.
- I am a zoo keeper of "computer science" :)☆10Jul 29, 2026Updated 3 months ago
- Power Plant ML Pipeline Application - Apache Spark☆12Dec 12, 2016Updated 9 years ago
- Marshmallow serializer integration with pyspark☆12Dec 29, 2023Updated 2 years ago
- Example project for consuming AWS Kinesis streamming and save data on Amazon Redshift using Apache Spark☆11May 22, 2018Updated 8 years ago
- Pytorch--使用伪标签训练efficientNet模型☆11Dec 28, 2019Updated 6 years ago
- Use aws-emr and aws-redshift to analyse dataset of adult census of USA☆13Sep 11, 2020Updated 6 years ago
- Analyzing Big Data with Amazon EMR☆12Sep 14, 2020Updated 6 years ago
- This repo is mostly created for pyspark and hive related interview questions.☆64Jan 6, 2026Updated 9 months ago
- Counting Tweets Per User in Real-Time☆43Jul 28, 2017Updated 9 years ago
- Managed Database hosting by DigitalOcean • AdPostgreSQL, MySQL, MongoDB, Kafka, Valkey, and OpenSearch available. Automatically scale up storage and focus on building your apps.
- Python Tensorflow 2 scripts for detecting objects of any class in an image without knowing their label.☆16Sep 18, 2021Updated 5 years ago
- A data pipeline project build on databricks and azure to demostrate lifecycle of a cloud data project.☆18Jan 12, 2022Updated 4 years ago
- https://www.kaggle.com/c/nbme-score-clinical-patient-notes☆10Sep 1, 2022Updated 4 years ago
- Methods for mapping proteomics data on 3D protein structure.☆15Jan 18, 2020Updated 6 years ago
- Dockerized python app to measure air quality, temperature and more using a raspberry pi + sensor☆15Jul 7, 2026Updated 3 months ago
- This is a pipeline of an ETL application in GCP with open airport code data, which you can find here: https://datahub.io/core/airport-cod…☆18Nov 15, 2021Updated 4 years ago
- Jupyter Notebook showing how to process Telecom datasets using PySpark (SparkSQL and DataFrames) and plotting the results using Matplotli…☆17Dec 3, 2018Updated 7 years ago