NIH-HPC / spark-slurmLinks
Tool for running/managing ad hoc spark clusters on a Slurm cluster
☆25Updated 3 years ago
Alternatives and similar repositories for spark-slurm
Users that are interested in spark-slurm are comparing it to the libraries listed below
Sorting:
- ☆100Updated 11 months ago
- Magpie contains a number of scripts for running Big Data software in HPC environments, including Hadoop and Spark. There is support for L…☆198Updated 7 months ago
- Documented examples of Jupyterhub deployment in HPC settings☆36Updated last year
- Slurm on Google Cloud Platform☆188Updated last year
- Custom Spawner for Jupyterhub to start servers in batch scheduled systems☆198Updated last week
- orchestration for singularity containers (under development)☆76Updated last year
- ☆13Updated 9 years ago
- %conda magic for IPython☆28Updated 8 years ago
- Local filesystem registry for containers (intended for HPC) using Lmod or Environment Modules. Works for users and admins.☆125Updated 3 months ago
- A command line interface to Globus☆82Updated this week
- Mechanism for runtime configuration of spawners for JupyterHub☆65Updated last year
- Spawn JupyterHub single-user servers with ssh☆27Updated 2 years ago
- Container tools for scientific computing! Docs at https://singularityhub.github.io/singularityhub-docs☆69Updated 3 years ago
- Build and deploy Singularity containers to GitHub releases, and pull with the singularity-hpc client☆17Updated last year
- Jupyter plugin that provides a tab for TACC Lmod (https://github.com/TACC/Lmod)☆33Updated last month
- A demo and tutorial for how to use git and singularity to make your research more portable and reproducible☆26Updated 8 years ago
- IPython magic for SLURM.☆69Updated 5 years ago
- NEWT - REST API for HPC☆11Updated 6 years ago
- Slurm Docker Container on CentOS 7☆88Updated last year
- ☆94Updated 6 years ago
- DRMAA for Slurm: Implementation of the DRMAA C bindings for Slurm☆49Updated 6 months ago
- Batch Connect - OSC Code Server☆18Updated last month
- Direct Acyclic Graph (DAG) Workflow Submitter for SLURM queueing system☆16Updated last year
- Run singularity containers on the Comet Supercomputer at San Diego Supercomputer Center☆18Updated 5 years ago
- Deploy Dask on DRMAA clusters☆40Updated 4 years ago
- Images for Saturn Cloud☆13Updated this week
- Slides and materials for workshop at University of Michigan MICDE☆19Updated 9 years ago
- An opinionated open source deployment of jupyterhub based on an Slurm job scheduler.☆28Updated 11 months ago
- Collection of tutorials for using Shifter to bring containers to HPC☆27Updated 6 years ago
- server for storage and management of singularity images☆104Updated last year