jacobceles / intro-to-colab-pyspark-emr
A tutorial that helps Big Data Engineers ramp up faster by getting familiar with PySpark dataframes and functions. It also covers topics like EMR sizing, Google Colaboratory, fine-tuning PySpark jobs, and much more.
☆18Updated 3 years ago
Related projects ⓘ
Alternatives and complementary repositories for intro-to-colab-pyspark-emr
- This is a repository for the Duke University Cloud Computing course project on Serveless Data Engineering Pipeline. For this project, I r…☆19Updated 3 years ago
- ☆18Updated 3 years ago
- Use Multiple Linear Regression, Python, Pandas, and Matplotlib to analyze the lifetime value and the key factors of the ‘Telco Customer C…☆10Updated 4 years ago
- PySpark Tutorial for Beginners on Google Colab: Hands-On Guide☆16Updated 4 years ago
- Demo on how to use Prefect with Docker☆26Updated 2 years ago
- Instant search for and access to many datasets in Pyspark.☆34Updated 2 years ago
- Deploy A/B testing infrastructure in a containerized microservice architecture for Machine Learning applications.☆39Updated last year
- ☆21Updated last year
- ☆19Updated 6 years ago
- Best practices for engineering ML pipelines.☆37Updated 2 years ago
- ☆25Updated 2 years ago
- Implementation of Spark code in Jupyter notebook. Topics include: RDDs and DataFrame, exploratory data analysis (EDA), handling multiple …☆28Updated 4 years ago
- Operations Research Algorithms☆17Updated 8 months ago
- ☆19Updated 3 years ago
- Powerful rapid automatic EDA and feature engineering library with a very easy to use API 🌟☆53Updated 2 years ago
- Creating a Gradio user interface to predict the sentiment of a tweet☆12Updated 2 years ago
- Blog post on ETL pipelines with Airflow☆23Updated 4 years ago
- The goal of this project is to offer an AWS EMR template using Spot Fleet and On-Demand Instances that you can use quickly. Just focus on…☆26Updated 2 years ago
- Here I will be exploring various tools and methods that are used in data engineering process with Python.☆22Updated 3 years ago
- an end-to-end data pipeline extracting music listening habits and producing an insightful dashboard☆14Updated 7 months ago
- Content for a talk on "The wonderful world of data quality tools in Python"☆18Updated 3 years ago
- PySpark Cheatsheet☆35Updated last year
- A guide to show you how to import data for ETL☆20Updated last year
- An example MLFlow project☆48Updated 2 years ago
- Full stack data engineering tools and infrastructure set-up☆44Updated 3 years ago
- ☆11Updated 2 years ago
- This repo contains all the material developed during the 9-week bootcamp provided by DPhi in colaboration with DataTalks Club☆21Updated 2 years ago
- ☆13Updated 3 years ago
- Create Interactive Dashboards with Streamlit and Python Coursera☆10Updated 4 years ago
- reating a modern data pipeline using a combination of Terraform, AWS Lambda and S3, Snowflake, DBT, Mage AI, and Dash.☆12Updated last year