glennklockwood / myhadoop
Framework for deploying Hadoop clusters on traditional HPC from userland
☆46Updated 6 years ago
Related projects ⓘ
Alternatives and complementary repositories for myhadoop
- Magpie contains a number of scripts for running Big Data software in HPC environments, including Hadoop and Spark. There is support for L…☆196Updated 11 months ago
- hanythingondemand provides a set of scripts to easily set up an ad-hoc Hadoop cluster through PBS jobs☆12Updated 5 years ago
- gather and plot data about Slurm scheduling and job statistics☆50Updated 10 years ago
- TACC Stats is an automated resource-usage monitoring and analysis package.☆42Updated last month
- XALT: System tracking of users codes on clusters☆43Updated this week
- launching and controlling spark on hpc clusters☆23Updated 2 years ago
- Deploy Dask on DRMAA clusters☆40Updated 3 years ago
- Jupyter plugin that provides a tab for TACC Lmod (https://github.com/TACC/Lmod)☆28Updated 2 weeks ago
- Documented examples of Jupyterhub deployment in HPC settings☆36Updated 5 months ago
- ☆28Updated 5 years ago
- Run singularity containers on the Comet Supercomputer at San Diego Supercomputer Center☆18Updated 4 years ago
- Custom Spawner for Jupyterhub to start slurm jobs when users log in☆24Updated 2 years ago
- Custom Spawner for Jupyterhub to start servers in batch scheduled systems☆190Updated 2 weeks ago
- Grand Unified File-Index☆46Updated 3 weeks ago
- High-level python wapper for the Sun Grid Engine (SGE) using DRMAA and ZMQ☆21Updated 10 years ago
- The configuration and scripts for using spark within a Sun Grid Engine environment☆24Updated 10 years ago
- ☆58Updated 2 years ago
- Machine Learning Toolkit for Extreme Scale (MaTEx)☆111Updated 6 years ago
- Data Accelerator: Creates a burst buffer from generic hardware and integrates it with Slurm https://www.hpc.cam.ac.uk/research/data-acc h…☆17Updated last year
- ☆19Updated 3 years ago
- This repository is now stale. You should be looking at the open-mpi/ompi repository instead.☆35Updated 8 years ago
- Deploy Dask using MPI4Py☆52Updated last month
- ☆95Updated last month
- Slides and materials for workshop at University of Michigan MICDE☆19Updated 8 years ago
- Harvard Extensions for Lmod deployment☆28Updated this week
- Tool for running/managing ad hoc spark clusters on a Slurm cluster☆22Updated 2 years ago
- MarFS provides a scalable near-POSIX file system by using one or more POSIX file systems as a scalable metadata component and one or more…☆97Updated 2 weeks ago
- Several utilities to aid in the use, administration, and management of PBS variants (including OpenPBS, PBS Pro, and TORQUE).☆25Updated 2 years ago
- Scalable dynamic library and python loading in HPC environments☆96Updated this week