run-ai / genvLinks
GPU environment and cluster management with LLM support
☆611Updated last year
Alternatives and similar repositories for genv
Users that are interested in genv are comparing it to the libraries listed below
Sorting:
- A top-like tool for monitoring GPUs in a cluster☆84Updated last year
- ☆222Updated this week
- PyTriton is a Flask/FastAPI-like interface that simplifies Triton's deployment in Python environments.☆801Updated 4 months ago
- Module, Model, and Tensor Serialization/Deserialization☆241Updated 2 weeks ago
- KAI Scheduler is an open source Kubernetes Native scheduler for AI workloads at large scale☆656Updated last week
- Module to Automatically maximize the utilization of GPU resources in a Kubernetes cluster through real-time dynamic partitioning and elas…☆664Updated last year
- Distributed Model Serving Framework☆173Updated 3 weeks ago
- markdown docs☆89Updated 2 weeks ago
- TorchX is a universal job launcher for PyTorch applications. TorchX is designed to have fast iteration time for training/research and sup…☆368Updated last week
- Triton Model Analyzer is a CLI tool to help with better understanding of the compute and memory requirements of the Triton Inference Serv…☆477Updated 2 weeks ago
- NVIDIA Data Center GPU Manager (DCGM) is a project for gathering telemetry and measuring the health of NVIDIA GPUs☆532Updated last month
- Practical GPU Sharing Without Memory Size Constraints☆271Updated 2 months ago
- MIG Partition Editor for NVIDIA GPUs☆201Updated last week
- Controller for ModelMesh☆232Updated 2 weeks ago
- Tools for building GPU clusters☆1,365Updated 2 months ago
- A simple yet powerful tool to turn traditional container/OS images into unprivileged sandboxes.☆777Updated 6 months ago
- aim-mlflow integration☆214Updated last year
- ClearML Fractional GPU - Run multiple containers on the same GPU with driver level memory limitation ✨ and compute time-slicing☆78Updated 10 months ago
- The simplest way to serve AI/ML models in production☆1,015Updated this week
- GPUd automates monitoring, diagnostics, and issue identification for GPUs☆377Updated this week
- Triton Model Navigator is an inference toolkit designed for optimizing and deploying Deep Learning models with a focus on NVIDIA GPUs.☆205Updated 2 months ago
- Container plugin for Slurm Workload Manager☆347Updated 7 months ago
- ☆310Updated 10 months ago
- ClearML - Model-Serving Orchestration and Repository Solution☆151Updated 5 months ago
- Autoscale LLM (vLLM, SGLang, LMDeploy) inferences on Kubernetes (and others)☆269Updated last year
- RayLLM - LLMs on Ray (Archived). Read README for more info.☆1,260Updated 3 months ago
- Transform datasets at scale. Optimize datasets for fast AI model training.☆493Updated last week
- A library to analyze PyTorch traces.☆391Updated last week
- Dynamic Resource Allocation (DRA) for NVIDIA GPUs in Kubernetes☆377Updated last week
- CUDA checkpoint and restore utility☆345Updated 4 months ago