dsaidgovsg / python-sparkLinks
Docker image for a Python installation with Spark, Hadoop and Sqoop binaries
☆15Updated 8 years ago
Alternatives and similar repositories for python-spark
Users that are interested in python-spark are comparing it to the libraries listed below
Sorting:
- Just a boilerplate for PySpark and Flask☆36Updated 7 years ago
- Code to build a simple analytics data pipeline with Python☆102Updated 8 years ago
- 🚨 Simple, self-contained fraud detection system built with Apache Kafka and Python☆89Updated 6 years ago
- (project & tutorial) dag pipeline tests + ci/cd setup☆90Updated 4 years ago
- One click deploy docker-compose with Kafka, Spark Streaming, Zeppelin UI and Monitoring (Grafana + Kafka Manager)☆120Updated 4 years ago
- Using Kafka-Python to illustrate a ML production pipeline☆112Updated 3 years ago
- Developed a data pipeline to automate data warehouse ETL by building custom airflow operators that handle the extraction, transformation,…☆89Updated 4 years ago
- An Airflow docker image preconfigured to work well with Spark and Hadoop/EMR☆175Updated 8 months ago
- Updated repository☆157Updated 4 years ago
- Repo for all my code on the articles I post on medium☆106Updated 3 years ago
- Repo that relates to the Medium blog 'Keeping your ML model in shape with Kafka, Airflow' and MLFlow'☆121Updated 2 years ago
- MLFlow Spark Summit 2019 Presentation☆67Updated 6 years ago
- Repository used for Spark Trainings☆54Updated 2 years ago
- Airflow training for the crunch conf☆105Updated 7 years ago
- Using Luigi to create a Machine Learning Pipeline using the Rossman Sales data from Kaggle☆33Updated 9 years ago
- ☆152Updated 7 years ago
- Various data stream/batch process demo with Apache Scala Spark 🚀☆11Updated 5 years ago
- Data Streaming Nanodegree (from Udacity) exercises, projects and their solutions☆17Updated 2 years ago
- LearningApacheSpark☆250Updated 2 years ago
- This repository shows a sample example to build, manage and orchestrate Machine Learning workflows using Amazon Sagemaker and Apache Airf…☆138Updated 4 years ago
- Use Airflow to move data from multiple MySQL databases to BigQuery☆100Updated 5 years ago
- Machine Learning model training & scalable deployment with Flask, Nginx & Gunicorn wrapped in a Docker Container☆21Updated 3 years ago
- Airflow basics tutorial☆397Updated 4 years ago
- PySpark Cookbook, published by Packt☆94Updated 3 years ago
- scaffold of Apache Airflow executing Docker containers☆85Updated 3 years ago
- ☆72Updated 6 years ago
- Data models, build data warehouses and data lakes, automate data pipelines, and worked with massive datasets.☆13Updated 6 years ago
- Demo of PySpark and Jupyter Notebook with the Jupyter Docker Stacks☆35Updated 5 years ago
- Jupyter notebooks for pyspark tutorials given at University☆110Updated 3 weeks ago
- Installation guide for Apache Spark + Hadoop on Mac/Linux☆60Updated 8 years ago