danielblazevski / airflow-pyspark-reddit
Example of using Airflow to schedule downloading data form S3 and launching spark jobs
☆15Updated 8 years ago
Alternatives and similar repositories for airflow-pyspark-reddit:
Users that are interested in airflow-pyspark-reddit are comparing it to the libraries listed below
- ☆28Updated 4 years ago
- Terraform module for a PostgreSQL-backed Apache Airflow instance☆24Updated 6 years ago
- A Getting Started Guide for developing and using Airflow Plugins☆94Updated 6 years ago
- Docker compose files for various kafka stacks☆32Updated 6 years ago
- This repo demonstrates how to load a sample Parquet formatted file from an AWS S3 Bucket. A python job will then be submitted to a Apach…☆19Updated 8 years ago
- Airflow workflow management platform chef cookbook.☆71Updated 5 years ago
- CLI tool to launch Spark jobs on AWS EMR☆67Updated last year
- An example PySpark project with pytest☆17Updated 7 years ago
- Multi docker container images for main Big Data Tools. (Hadoop, Spark, Kafka, HBase, Cassandra, Zookeeper, Zeppelin, Drill, Flink, Hive, …☆34Updated 2 months ago
- Sentiment Analysis of a Twitter Topic with Spark Structured Streaming☆55Updated 6 years ago
- ☆10Updated 6 years ago
- ☆54Updated 6 years ago
- Repo for building docker based airflow image. Containers support multiple features like writing logs to local or S3 folder and Initializi…☆32Updated 5 years ago
- Setup Apache Airflow on Kubernetes☆10Updated 6 years ago
- Source code for 'PySpark Recipes' by Raju Kumar Mishra☆25Updated 5 years ago
- Some AWS EMR examples☆16Updated 7 years ago
- Data validation library for PySpark 3.0.0☆34Updated 2 years ago
- Basic tutorial of using Apache Airflow☆36Updated 6 years ago
- A toolset to streamline running spark python on EMR☆20Updated 8 years ago
- Spark Application UI extension for JupyterLab☆10Updated 3 years ago
- A cookiecutter template for Apache Spark applications written in Scala☆10Updated 6 years ago
- Python and Dask: Scaling the Dataframe☆40Updated 3 years ago
- Dump mysql tables to s3, and parse them☆31Updated 10 years ago
- ☆26Updated 4 years ago
- Realtime social media data analytics with Apache Spark, Python, Kafka, Pandas, etc☆51Updated 8 years ago
- Code to build a simple analytics data pipeline with Python☆102Updated 7 years ago
- Skeleton project for Apache Airflow training participants to work on.☆16Updated 4 years ago
- REST-like API exposing Airflow data and operations☆61Updated 6 years ago
- Ansible role to deploy and configure Airflow☆41Updated 3 weeks ago
- Python library for deploying models built using Python to Alteryx Promote.☆15Updated 3 years ago