NVIDIA / nim-deployLinks
A collection of YAML files, Helm Charts, Operator code, and guides to act as an example reference implementation for NVIDIA NIM deployment.
☆221Updated this week
Alternatives and similar repositories for nim-deploy
Users that are interested in nim-deploy are comparing it to the libraries listed below
Sorting:
- Accelerate your Gen AI with NVIDIA NIM and NVIDIA AI Workbench☆202Updated 9 months ago
- Infrastructure as code for GPU accelerated managed Kubernetes clusters.☆57Updated 9 months ago
- An Operator for deployment and maintenance of NVIDIA NIMs and NeMo microservices in a Kubernetes environment.☆146Updated this week
- Run cloud native workloads on NVIDIA GPUs☆223Updated 2 weeks ago
- The NVIDIA NeMo Agent Toolkit UI streamlines interacting with NeMo Agent Toolkit workflows in an easy-to-use web application.☆74Updated this week
- markdown docs☆93Updated this week
- GenAI components at micro-service level; GenAI service composer to create mega-service☆193Updated 2 weeks ago
- AI on GKE is a collection of examples, best-practices, and prebuilt solutions to help build, deploy, and scale AI Platforms on Google Kub…☆327Updated 7 months ago
- An NVIDIA AI Workbench example project for fine-tuning a Mistral 7B model☆69Updated last year
- ☆280Updated this week
- ☆184Updated last week
- Repository for open inference protocol specification☆64Updated 8 months ago
- This NVIDIA RAG blueprint serves as a reference solution for a foundational Retrieval Augmented Generation (RAG) pipeline.☆460Updated this week
- Provides end-to-end model development pipelines for LLMs and Multimodal models that can be launched on-prem or cloud-native.☆510Updated 9 months ago
- Training and inference on AWS Trainium and Inferentia chips.☆258Updated last week
- Helm charts for the KubeRay project☆59Updated 2 months ago
- Containerization and cloud native suite for OPEA☆74Updated last month
- Route LLM requests to the best model for the task at hand.☆173Updated 3 weeks ago
- Controller for ModelMesh☆242Updated 7 months ago
- Self-host LLMs with vLLM and BentoML☆168Updated 2 weeks ago
- Hugging Face Deep Learning Containers (DLCs) for Google Cloud☆162Updated last week
- JetStream is a throughput and memory optimized engine for LLM inference on XLA devices, starting with TPUs (and GPUs in future -- PRs wel…☆404Updated last month
- A high-throughput and memory-efficient inference and serving engine for LLMs☆24Updated 3 months ago
- Distributed Model Serving Framework☆183Updated 4 months ago
- ☆166Updated 2 weeks ago
- A toolkit for discovering cluster network topology.☆96Updated this week
- Pretrain, finetune and serve LLMs on Intel platforms with Ray☆131Updated 4 months ago
- Tutorial for building LLM router☆244Updated last year
- This is suite of the hands-on training materials that shows how to scale CV, NLP, time-series forecasting workloads with Ray.☆451Updated last year
- Module, Model, and Tensor Serialization/Deserialization☆286Updated 5 months ago