NIH-HPC / spark-slurmLinks
Tool for running/managing ad hoc spark clusters on a Slurm cluster
☆25Updated 3 years ago
Alternatives and similar repositories for spark-slurm
Users that are interested in spark-slurm are comparing it to the libraries listed below
Sorting:
- ☆100Updated last year
- Magpie contains a number of scripts for running Big Data software in HPC environments, including Hadoop and Spark. There is support for L…☆198Updated 8 months ago
- Slurm on Google Cloud Platform☆188Updated last year
- Documented examples of Jupyterhub deployment in HPC settings☆36Updated last year
- SLURM Tools and UBiLities☆74Updated 3 years ago
- Container tools for scientific computing! Docs at https://singularityhub.github.io/singularityhub-docs☆69Updated 3 years ago
- Local filesystem registry for containers (intended for HPC) using Lmod or Environment Modules. Works for users and admins.☆127Updated 5 months ago
- DRMAA for Slurm: Implementation of the DRMAA C bindings for Slurm☆49Updated last month
- orchestration for singularity containers (under development)☆76Updated last year
- Slurm Docker Container on CentOS 7☆88Updated last year
- Jupyter plugin that provides a tab for TACC Lmod (https://github.com/TACC/Lmod)☆33Updated 3 months ago
- Custom Spawner for Jupyterhub to start servers in batch scheduled systems☆198Updated 3 weeks ago
- server for storage and management of singularity images☆105Updated last year
- IPython magic for SLURM.☆69Updated 5 years ago
- files and instructions for creating and using example containers from the sylabs.io blog☆105Updated 2 years ago
- A simple utility for executing multiple sequential or multi-threaded applications in a single multi-node batch job☆64Updated last year
- For installing and configuring SLURM - Simple Linux Utility for Resource Management☆36Updated last year
- A few utilities for use on a SLURM cluster☆43Updated 2 months ago
- definition files and wrapper scripts used by NIH HPC staff to install user-facing apps on the Biowulf cluster☆32Updated 4 years ago
- Create clusters of VMs on the cloud and configure them with Ansible.☆338Updated 2 years ago
- Batch Connect - OSC Code Server☆18Updated 3 months ago
- Script for doing Slurm Calculations☆12Updated 7 months ago
- dead Simple Queue - Slurm arrays for files with jobs in them.☆24Updated 3 years ago
- Mechanism for runtime configuration of spawners for JupyterHub☆65Updated last year
- NEWT - REST API for HPC☆11Updated 6 years ago
- Terraform modules to replicate the HPC user experience in the cloud☆157Updated this week
- Ansible role for installing and managing the Slurm Workload Manager☆108Updated 6 months ago
- Build and deploy Singularity containers to GitHub releases, and pull with the singularity-hpc client☆17Updated 2 years ago
- Scripts and Ansible Playbooks for building an HPC-style resource in Jetstream☆19Updated last year
- Collection of tutorials for using Shifter to bring containers to HPC☆27Updated 6 years ago