NVIDIA / nim-deployLinks
A collection of YAML files, Helm Charts, Operator code, and guides to act as an example reference implementation for NVIDIA NIM deployment.
☆191Updated this week
Alternatives and similar repositories for nim-deploy
Users that are interested in nim-deploy are comparing it to the libraries listed below
Sorting:
- Accelerate your Gen AI with NVIDIA NIM and NVIDIA AI Workbench☆180Updated 4 months ago
- An Operator for deployment and maintenance of NVIDIA NIMs and NeMo microservices in a Kubernetes environment.☆125Updated this week
- Infrastructure as code for GPU accelerated managed Kubernetes clusters.☆55Updated 4 months ago
- ☆241Updated this week
- Run cloud native workloads on NVIDIA GPUs☆196Updated 3 weeks ago
- GenAI components at micro-service level; GenAI service composer to create mega-service☆172Updated this week
- Repository for open inference protocol specification☆59Updated 4 months ago
- markdown docs☆93Updated 2 weeks ago
- Evaluation, benchmark, and scorecard, targeting for performance on throughput and latency, accuracy on popular evaluation harness, safety…☆37Updated 3 weeks ago
- Controller for ModelMesh☆238Updated 3 months ago
- Containerization and cloud native suite for OPEA☆70Updated 3 weeks ago
- ☆165Updated last week
- Distributed Model Serving Framework☆177Updated 3 weeks ago
- Self-host LLMs with vLLM and BentoML☆145Updated this week
- JetStream is a throughput and memory optimized engine for LLM inference on XLA devices, starting with TPUs (and GPUs in future -- PRs wel…☆375Updated 3 months ago
- This NVIDIA RAG blueprint serves as a reference solution for a foundational Retrieval Augmented Generation (RAG) pipeline.☆255Updated 2 weeks ago
- Hugging Face Deep Learning Containers (DLCs) for Google Cloud☆153Updated 4 months ago
- Evaluate and Enhance Your LLM Deployments for Real-World Inference Needs☆566Updated this week
- Kubernetes enhancements for Network Topology Aware Gang Scheduling & Autoscaling☆46Updated this week
- KAI Scheduler is an open source Kubernetes Native scheduler for AI workloads at large scale☆801Updated this week
- Helm charts for the KubeRay project☆50Updated 2 months ago
- ☆29Updated 3 weeks ago
- ☆144Updated last week
- This repo contains documents of the OPEA project☆44Updated 3 weeks ago
- Kubernetes Operator, ansible playbooks, and production scripts for large-scale AIStore deployments on Kubernetes.☆109Updated last week
- Triton CLI is an open source command line interface that enables users to create, deploy, and profile models served by the Triton Inferen…☆67Updated last week
- Pretrain, finetune and serve LLMs on Intel platforms with Ray☆132Updated last week
- An NVIDIA AI Workbench example project for fine-tuning a Mistral 7B model☆60Updated last year
- Provides end-to-end model development pipelines for LLMs and Multimodal models that can be launched on-prem or cloud-native.☆511Updated 4 months ago
- IBM development fork of https://github.com/huggingface/text-generation-inference☆61Updated 4 months ago