NVIDIA / nim-deployLinks
A collection of YAML files, Helm Charts, Operator code, and guides to act as an example reference implementation for NVIDIA NIM deployment.
☆216Updated last week
Alternatives and similar repositories for nim-deploy
Users that are interested in nim-deploy are comparing it to the libraries listed below
Sorting:
- Accelerate your Gen AI with NVIDIA NIM and NVIDIA AI Workbench☆192Updated 7 months ago
- An Operator for deployment and maintenance of NVIDIA NIMs and NeMo microservices in a Kubernetes environment.☆140Updated this week
- Infrastructure as code for GPU accelerated managed Kubernetes clusters.☆57Updated 7 months ago
- Run cloud native workloads on NVIDIA GPUs☆208Updated last month
- ☆268Updated last week
- Repository for open inference protocol specification☆60Updated 6 months ago
- markdown docs☆92Updated last month
- AI on GKE is a collection of examples, best-practices, and prebuilt solutions to help build, deploy, and scale AI Platforms on Google Kub…☆326Updated 5 months ago
- This NVIDIA RAG blueprint serves as a reference solution for a foundational Retrieval Augmented Generation (RAG) pipeline.☆385Updated this week
- GenAI components at micro-service level; GenAI service composer to create mega-service☆187Updated last week
- Evaluate and Enhance Your LLM Deployments for Real-World Inference Needs☆730Updated this week
- ☆176Updated this week
- Route LLM requests to the best model for the task at hand.☆137Updated 3 weeks ago
- Controller for ModelMesh☆242Updated 5 months ago
- Helm charts for the KubeRay project☆59Updated 2 weeks ago
- Pretrain, finetune and serve LLMs on Intel platforms with Ray☆130Updated 2 months ago
- The NVIDIA AIQToolkit UI streamlines interacting with AIQToolkit workflows in an easy-to-use web application.☆55Updated this week
- An NVIDIA AI Workbench example project for fine-tuning a Mistral 7B model☆66Updated last year
- A toolkit for discovering cluster network topology.☆84Updated this week
- A high-throughput and memory-efficient inference and serving engine for LLMs☆23Updated last month
- Self-host LLMs with vLLM and BentoML☆161Updated last week
- Distributed Model Serving Framework☆179Updated 2 months ago
- Kubernetes Operator, ansible playbooks, and production scripts for large-scale AIStore deployments on Kubernetes.☆117Updated this week
- Evaluation, benchmark, and scorecard, targeting for performance on throughput and latency, accuracy on popular evaluation harness, safety…☆38Updated last month
- KAI Scheduler is an open source Kubernetes Native scheduler for AI workloads at large scale☆956Updated this week
- Containerization and cloud native suite for OPEA☆72Updated 2 months ago
- ☆29Updated 3 months ago
- JetStream is a throughput and memory optimized engine for LLM inference on XLA devices, starting with TPUs (and GPUs in future -- PRs wel…☆392Updated 5 months ago
- LLMPerf is a library for validating and benchmarking LLMs☆1,057Updated 11 months ago
- ☆157Updated this week