Alternatives and similar repositories for pyspark-testing:

Users that are interested in pyspark-testing are comparing it to the libraries listed below

• kawadia / pyspark.test
  Example unit tests for Apache Spark Python scripts using the py.test framework
  ☆84Updated 9 years ago
• jwplayer / sparksteps
  CLI tool to launch Spark jobs on AWS EMR
  ☆67Updated last year
• malexer / pytest-spark
  pytest plugin to run the tests with support of pyspark
  ☆86Updated last week
• geosolutions-it / evo-odas
  Code Repository for the EVO-ODAS
  ☆31Updated 7 years ago
• industrydive / fileflow
  Airflow plugin to transfer arbitrary files between operators
  ☆78Updated 6 years ago
• aws-samples / aws-concurrent-data-orchestration-pipeline-emr-livy
  This code demonstrates the architecture featured on the AWS Big Data blog (https://aws.amazon.com/blogs/big-data/ ) which creates a concu…
  ☆75Updated 6 years ago
• bahchis / airflow-cookbook
  Airflow workflow management platform chef cookbook.
  ☆71Updated 5 years ago
• jrderuiter / airflow-fs
  Composable filesystem hooks and operators for Apache Airflow.
  ☆17Updated 3 years ago
• zero323 / pyspark-stubs
  Apache (Py)Spark type annotations (stub files).
  ☆116Updated 2 years ago
• MrPowers / gill
  An example PySpark project with pytest
  ☆17Updated 7 years ago
• dask / dask-ec2
  Start a cluster in EC2 for dask.distributed
  ☆106Updated 4 years ago
• BasPH / pylint-airflow
  Pylint plugin for static code analysis on Airflow code
  ☆93Updated 4 years ago
• michaelosthege / fairflow
  Functional Airflow DAG definitions.
  ☆38Updated 7 years ago
• sparklingpandas / sparklingml
  Machine Learning Pipeline Stages for Spark (exposed in Scala/Java + Python)
  ☆74Updated last year
• ShopRunner / stork
  Make your libraries magically appear in Databricks.
  ☆47Updated last year
• svenkreiss / pysparkling
  A pure Python implementation of Apache Spark's RDD and DStream interfaces.
  ☆268Updated 6 months ago
• dask / dask-yarn
  Deploy dask on YARN clusters
  ☆69Updated 7 months ago
• julioasotodv / spark-df-profiling
  Create HTML profiling reports from Apache Spark DataFrames
  ☆195Updated 5 years ago
• danielvdende / data-testing-with-airflow
  ☆198Updated last year
• awslabs / aws-iot-analytics-notebook-containers
  An extension for Jupyter notebooks that allows running notebooks inside a Docker container and converting them to runnable Docker images.
  ☆28Updated last year
• datitran / emr-bootstrap-pyspark
  Quickstart PySpark with Anaconda on AWS/EMR
  ☆53Updated 8 years ago
• dask / knit
  Deprecated, please use https://github.com/jcrist/skein or https://github.com/dask/dask-yarn instead
  ☆52Updated 6 years ago
• hellonarrativ / spectrify
  Export Redshift data and convert to Parquet for use with Redshift Spectrum or other data warehouses.
  ☆116Updated 2 years ago
• hammerlab / dask-distributed-on-kubernetes
  Deploy dask-distributed on google container engine using kubernetes
  ☆40Updated 5 years ago
• rdblue / jupyter-zeppelin
  Conversion utility from Zeppelin notes to Jupyter notebooks.
  ☆44Updated 5 years ago
• ychantit / airflow_aws_utils
  A collection of airflow sample workflows for data processing on aws
  ☆12Updated 7 years ago
• laughingman7743 / PyAthenaJDBC
  PyAthenaJDBC is an Amazon Athena JDBC driver wrapper for the Python DB API 2.0 (PEP 249).
  ☆95Updated last year
• seahboonsiew / pyspark-csv
  An external PySpark module that works like R's read.csv or Panda's read_csv, with automatic type inference and null value handling. Parse…
  ☆90Updated 9 years ago
• zero323 / pyspark-asyncactions
  Asynchronous actions for PySpark
  ☆47Updated 3 years ago
• nteract / coffee_boat
  ☕⛵WIP PySpark dependency management
  ☆22Updated 6 years ago