NVIDIA / nim-deployLinks
A collection of YAML files, Helm Charts, Operator code, and guides to act as an example reference implementation for NVIDIA NIM deployment.
☆190Updated last week
Alternatives and similar repositories for nim-deploy
Users that are interested in nim-deploy are comparing it to the libraries listed below
Sorting:
- Accelerate your Gen AI with NVIDIA NIM and NVIDIA AI Workbench☆179Updated 5 months ago
- An Operator for deployment and maintenance of NVIDIA NIMs and NeMo microservices in a Kubernetes environment.☆129Updated last week
- Infrastructure as code for GPU accelerated managed Kubernetes clusters.☆55Updated 5 months ago
- Run cloud native workloads on NVIDIA GPUs☆198Updated last week
- ☆255Updated this week
- Repository for open inference protocol specification☆59Updated 4 months ago
- Evaluate and Enhance Your LLM Deployments for Real-World Inference Needs☆596Updated last week
- This NVIDIA RAG blueprint serves as a reference solution for a foundational Retrieval Augmented Generation (RAG) pipeline.☆280Updated last week
- Helm charts for the KubeRay project☆51Updated 2 months ago
- ☆168Updated last week
- Controller for ModelMesh☆237Updated 3 months ago
- AI on GKE is a collection of examples, best-practices, and prebuilt solutions to help build, deploy, and scale AI Platforms on Google Kub…☆324Updated 3 months ago
- Provides end-to-end model development pipelines for LLMs and Multimodal models that can be launched on-prem or cloud-native.☆507Updated 5 months ago
- Distributed Model Serving Framework☆178Updated last week
- markdown docs☆93Updated last week
- Triton CLI is an open source command line interface that enables users to create, deploy, and profile models served by the Triton Inferen…☆68Updated 3 weeks ago
- OpenTelemetry Instrumentation for AI Observability☆632Updated this week
- Self-host LLMs with vLLM and BentoML☆150Updated last week
- JetStream is a throughput and memory optimized engine for LLM inference on XLA devices, starting with TPUs (and GPUs in future -- PRs wel…☆379Updated 3 months ago
- The NVIDIA AIQToolkit UI streamlines interacting with AIQToolkit workflows in an easy-to-use web application.☆46Updated last week
- ☆470Updated last year
- LLMPerf is a library for validating and benchmarking LLMs☆1,019Updated 9 months ago
- ☆111Updated 8 months ago
- Containerization and cloud native suite for OPEA☆70Updated last month
- KAI Scheduler is an open source Kubernetes Native scheduler for AI workloads at large scale☆838Updated this week
- ☆49Updated 2 months ago
- Pretrain, finetune and serve LLMs on Intel platforms with Ray☆132Updated 2 weeks ago
- ☆146Updated last week
- An NVIDIA AI Workbench example project for Retrieval Augmented Generation (RAG)☆340Updated last month
- GenAI components at micro-service level; GenAI service composer to create mega-service☆178Updated last week