NVIDIA / nim-deployLinks
A collection of YAML files, Helm Charts, Operator code, and guides to act as an example reference implementation for NVIDIA NIM deployment.
☆185Updated 2 weeks ago
Alternatives and similar repositories for nim-deploy
Users that are interested in nim-deploy are comparing it to the libraries listed below
Sorting:
- Accelerate your Gen AI with NVIDIA NIM and NVIDIA AI Workbench☆174Updated 2 months ago
- An Operator for deployment and maintenance of NVIDIA NIMs and NeMo microservices in a Kubernetes environment.☆117Updated this week
- Repository for open inference protocol specification☆57Updated 2 months ago
- ☆228Updated this week
- Run cloud native workloads on NVIDIA GPUs☆186Updated 2 months ago
- AI on GKE is a collection of examples, best-practices, and prebuilt solutions to help build, deploy, and scale AI Platforms on Google Kub…☆322Updated 3 weeks ago
- GenAI components at micro-service level; GenAI service composer to create mega-service☆161Updated this week
- Evaluate and Enhance Your LLM Deployments for Real-World Inference Needs☆394Updated this week
- KAI Scheduler is an open source Kubernetes Native scheduler for AI workloads at large scale☆692Updated this week
- Containerization and cloud native suite for OPEA☆67Updated last week
- Helm charts for the KubeRay project☆48Updated last week
- ☆159Updated 2 weeks ago
- Controller for ModelMesh☆234Updated last month
- This NVIDIA RAG blueprint serves as a reference solution for a foundational Retrieval Augmented Generation (RAG) pipeline.☆150Updated this week
- Self-host LLMs with vLLM and BentoML☆133Updated 2 weeks ago
- Provides end-to-end model development pipelines for LLMs and Multimodal models that can be launched on-prem or cloud-native.☆505Updated 2 months ago
- Kubernetes Operator, ansible playbooks, and production scripts for large-scale AIStore deployments on Kubernetes.☆102Updated last week
- Triton CLI is an open source command line interface that enables users to create, deploy, and profile models served by the Triton Inferen…☆64Updated last month
- Distributed Model Serving Framework☆173Updated last month
- Route LLM requests to the best model for the task at hand.☆78Updated 2 weeks ago
- Pretrain, finetune and serve LLMs on Intel platforms with Ray☆129Updated last week
- JetStream is a throughput and memory optimized engine for LLM inference on XLA devices, starting with TPUs (and GPUs in future -- PRs wel…☆354Updated last month
- markdown docs☆89Updated this week
- This is suite of the hands-on training materials that shows how to scale CV, NLP, time-series forecasting workloads with Ray.☆416Updated last year
- Cluster Toolkit is an open-source software offered by Google Cloud which makes it easy for customers to deploy AI/ML and HPC environments…☆272Updated this week
- MIG Partition Editor for NVIDIA GPUs☆202Updated this week
- Helm charts for llm-d☆50Updated last week
- AI Inference Operator for Kubernetes. The easiest way to serve ML models in production. Supports VLMs, LLMs, embeddings, and speech-to-te…☆1,016Updated last week
- Easy, fast and very cheap training and inference on AWS Trainium and Inferentia chips.☆233Updated this week
- Generative AI Examples is a collection of GenAI examples such as ChatQnA, Copilot, which illustrate the pipeline capabilities of the Open…☆648Updated this week