NVIDIA / nim-deploy
A collection of YAML files, Helm Charts, Operator code, and guides to act as an example reference implementation for NVIDIA NIM deployment.
☆157Updated last week
Alternatives and similar repositories for nim-deploy:
Users that are interested in nim-deploy are comparing it to the libraries listed below
- Accelerate your Gen AI with NVIDIA NIM and NVIDIA AI Workbench☆146Updated 3 weeks ago
- ☆159Updated this week
- An Operator for deployment and maintenance of NVIDIA NIMs and NeMo microservices in a Kubernetes environment.☆82Updated this week
- Infrastructure as code for GPU accelerated managed Kubernetes clusters.☆50Updated 2 months ago
- Easy, fast and very cheap training and inference on AWS Trainium and Inferentia chips.☆218Updated this week
- Run cloud native workloads on NVIDIA GPUs☆156Updated this week
- Self-host LLMs with vLLM and BentoML☆86Updated this week
- ☆102Updated last month
- Triton CLI is an open source command line interface that enables users to create, deploy, and profile models served by the Triton Inferen…☆54Updated this week
- ☆134Updated last week
- ☆40Updated last week
- Helm charts for the KubeRay project☆38Updated this week
- 🚀 Use NVIDIA NIMs with Haystack pipelines☆30Updated 5 months ago
- End-to-End LLM Guide☆101Updated 7 months ago
- Community-maintained Kubernetes config and Helm chart for Langfuse☆76Updated 3 weeks ago
- Evaluate and Enhance Your LLM Deployments for Real-World Inference Needs☆190Updated this week
- IBM development fork of https://github.com/huggingface/text-generation-inference☆59Updated 2 months ago
- JetStream is a throughput and memory optimized engine for LLM inference on XLA devices, starting with TPUs (and GPUs in future -- PRs wel…☆275Updated this week
- Using LlamaIndex with Ray for productionizing LLM applications☆71Updated last year
- An NVIDIA AI Workbench example project for fine-tuning a Mistral 7B model☆52Updated 8 months ago
- ☆224Updated this week
- Example code for AWS Neuron SDK developers building inference and training applications☆135Updated last week
- Tutorial for building LLM router☆181Updated 7 months ago
- Repository for open inference protocol specification☆46Updated 6 months ago
- Kubernetes Operator, ansible playbooks, and production scripts for large-scale AIStore deployments on Kubernetes.☆89Updated this week
- Collection of best practices, reference architectures, model training examples and utilities to train large models on AWS.☆244Updated this week
- Pretrain, finetune and serve LLMs on Intel platforms with Ray☆112Updated last week
- LLMPerf is a library for validating and benchmarking LLMs☆749Updated 2 months ago
- A tool to configure, launch and manage your machine learning experiments.☆117Updated this week