NVIDIA / nim-deploy
A collection of YAML files, Helm Charts, Operator code, and guides to act as an example reference implementation for NVIDIA NIM deployment.
☆177Updated last week
Alternatives and similar repositories for nim-deploy:
Users that are interested in nim-deploy are comparing it to the libraries listed below
- Accelerate your Gen AI with NVIDIA NIM and NVIDIA AI Workbench☆156Updated last month
- An Operator for deployment and maintenance of NVIDIA NIMs and NeMo microservices in a Kubernetes environment.☆92Updated this week
- Infrastructure as code for GPU accelerated managed Kubernetes clusters.☆55Updated 2 weeks ago
- ☆191Updated 2 weeks ago
- JetStream is a throughput and memory optimized engine for LLM inference on XLA devices, starting with TPUs (and GPUs in future -- PRs wel…☆315Updated this week
- Run cloud native workloads on NVIDIA GPUs☆168Updated last week
- ☆148Updated this week
- Evaluate and Enhance Your LLM Deployments for Real-World Inference Needs☆266Updated this week
- 🏋️ A unified multi-backend utility for benchmarking Transformers, Timm, PEFT, Diffusers and Sentence-Transformers with full support of O…☆294Updated 2 months ago
- GenAI components at micro-service level; GenAI service composer to create mega-service☆136Updated this week
- Triton CLI is an open source command line interface that enables users to create, deploy, and profile models served by the Triton Inferen…☆61Updated last month
- AI on GKE is a collection of examples, best-practices, and prebuilt solutions to help build, deploy, and scale AI Platforms on Google Kub…☆307Updated this week
- MIG Partition Editor for NVIDIA GPUs☆194Updated this week
- An NVIDIA AI Workbench example project for Retrieval Augmented Generation (RAG)☆310Updated last week
- Helm charts for the KubeRay project☆43Updated 2 weeks ago
- Containerization and cloud native suite for OPEA☆53Updated this week
- Kubernetes Operator, ansible playbooks, and production scripts for large-scale AIStore deployments on Kubernetes.☆93Updated this week
- Provides end-to-end model development pipelines for LLMs and Multimodal models that can be launched on-prem or cloud-native.☆501Updated last month
- Controller for ModelMesh☆228Updated 3 weeks ago
- This repo contains documents of the OPEA project☆32Updated last week
- Collection of reference workflows for building intelligent agents with NIMs☆152Updated 3 months ago
- ☆53Updated 7 months ago
- ☆243Updated this week
- KAI Scheduler is an open source Kubernetes Native scheduler for AI workloads at large scale☆483Updated this week
- Repository for open inference protocol specification☆53Updated 9 months ago
- This NVIDIA RAG blueprint serves as a reference solution for a foundational Retrieval Augmented Generation (RAG) pipeline.☆76Updated last month
- ☆105Updated 3 months ago
- Module, Model, and Tensor Serialization/Deserialization☆223Updated 2 months ago
- Recipes for reproducing training and serving benchmarks for large machine learning models using GPUs on Google Cloud.☆56Updated this week
- ☆45Updated 2 months ago