dsaidgovsg / python-spark

Related projects ⓘ

Alternatives and complementary repositories for python-spark

• mjhea0 / flask-spark-docker
  Just a boilerplate for PySpark and Flask
  ☆35Updated 6 years ago
• florimondmanca / kafka-fraud-detector
  🚨 Simple, self-contained fraud detection system built with Apache Kafka and Python
  ☆83Updated 5 years ago
• BogdanCojocar / medium-articles
  Repo for all my code on the articles I post on medium
  ☆105Updated 2 years ago
• mikeghen / airflow-tutorial
  Use Airflow to move data from multiple MySQL databases to BigQuery
  ☆99Updated 4 years ago
• NYUBigDataProject / SparkClean
  A Scalable Data Cleaning Library for PySpark.
  ☆26Updated 5 years ago
• redapt / pyspark-s3-parquet-example
  This repo demonstrates how to load a sample Parquet formatted file from an AWS S3 Bucket. A python job will then be submitted to a Apach…
  ☆19Updated 8 years ago
• igorman / data-engineering
  Challenge for those applying to the Software Engineer, Big Data position
  ☆34Updated 13 years ago
• dataquestio / analytics_pipeline
  Code to build a simple analytics data pipeline with Python
  ☆102Updated 7 years ago
• vergili / bigdata_tutorial
  ☆16Updated 6 years ago
• kadnan / Airflow-Tutorial
  Basic tutorial of using Apache Airflow
  ☆35Updated 6 years ago
• vim89 / datapipelines-essentials-python
  Simplified ETL process in Hadoop using Apache Spark. Has complete ETL pipeline for datalake. SparkSession extensions, DataFrame validatio…
  ☆53Updated last year
• ananthdurai / airflow-training
  Airflow training for the crunch conf
  ☆105Updated 6 years ago
• r-kells / prod-airflow
  Helping you get Airflow running in production.
  ☆9Updated 5 years ago
• deliveryhero / pyconde2019-airflow-ml-workshop
  PyConDE & PyData Berlin 2019 Airflow Workshop: Airflow for machine learning pipelines.
  ☆46Updated last year
• mdh266 / AirflowDataPipeline
  Example of an ETL Pipeline using Airflow
  ☆32Updated 7 years ago
• benjigoldberg / udacity-airflow
  Udacity Data Pipeline Exercises
  ☆15Updated 4 years ago
• edyoda / pyspark-tutorial
  PySpark Code for Hands-on Learners
  ☆114Updated 5 years ago
• Atreya22 / luigi_rosmann_sales
  Using Luigi to create a Machine Learning Pipeline using the Rossman Sales data from Kaggle
  ☆33Updated 8 years ago
• dsaidgovsg / airflow-pipeline
  An Airflow docker image preconfigured to work well with Spark and Hadoop/EMR
  ☆173Updated last year
• geoHeil / awesome-tools
  curated list of awesome tools and libraries for specific domains
  ☆35Updated this week
• hanee-shousha / TwitterStreaming
  Processing tweets using Spark Streaming and identifying top trending hashtags using a real-time simple dashboard
  ☆42Updated 2 years ago
• reisdebora / awesome-databricks
  A curated list of awesome Databricks resources, including Spark
  ☆14Updated 4 months ago
• SQLShark / MachineLearningFromModelToProduction
  This repo holds all the content for the Machine Learning training day
  ☆22Updated 2 years ago
• Yannael / kafka-sparkstreaming-cassandra
  Docker container for Kafka - Spark Streaming - Cassandra
  ☆97Updated 5 years ago
• rssanders3 / airflow-spark-operator-plugin
  A plugin to Apache Airflow to allow you to run Spark Submit Commands as an Operator
  ☆75Updated 5 years ago
• amwaleh / Simple-stream-Kafka
  This is a simple streaming application that utilises Kafka and Python
  ☆45Updated 5 years ago
• jpzk / cookiecutter-scala-spark
  A cookiecutter template for Apache Spark applications written in Scala
  ☆10Updated 5 years ago
• tfayyaz / awesome-airflow
  A curated list of all the awesome examples, articles, tutorials and videos for Apache Airflow.
  ☆96Updated 3 years ago