YH-Wu / Triton-Inference-Server-on-Kubernetes

Related projects ⓘ

Alternatives and complementary repositories for Triton-Inference-Server-on-Kubernetes

• triton-inference-server / dali_backend
  The Triton backend that allows running GPU-accelerated data pre-processing pipelines implemented in DALI's python API.
  ☆125Updated 2 weeks ago
• triton-inference-server / tensorrt_backend
  The Triton backend for TensorRT.
  ☆64Updated this week
• NVIDIA-AI-IOT / NVIDIA-Optical-Character-Detection-and-Recognition-Solution
  This repository provides optical character detection and recognition solution optimized on Nvidia devices.
  ☆55Updated last month
• triton-inference-server / onnxruntime_backend
  The Triton backend for the ONNX Runtime.
  ☆133Updated this week
• Bobo-y / triton_ensemble_model_demo
  triton server ensemble model demo
  ☆30Updated 2 years ago
• triton-inference-server / model_navigator
  Triton Model Navigator is an inference toolkit designed for optimizing and deploying Deep Learning models with a focus on NVIDIA GPUs.
  ☆185Updated 2 months ago
• kamalkraj / stable-diffusion-tritonserver
  Deploy stable diffusion model with onnx/tenorrt + tritonserver
  ☆123Updated last year
• NVIDIA-AI-IOT / nvidia-tao
  ☆82Updated 2 months ago
• NVIDIA-AI-IOT / yolov5_gpu_optimization
  This repository provides YOLOV5 GPU optimization sample
  ☆100Updated last year
• Curt-Park / mnist-fastapi-aio-triton
  Simple example of FastAPI + gRPC AsyncIO + Triton
  ☆59Updated 2 years ago
• NVIDIA-AI-IOT / tao-toolkit-triton-apps
  Sample app code for deploying TAO Toolkit trained models to Triton
  ☆84Updated 2 months ago
• triton-inference-server / backend
  Common source, scripts and utilities for creating Triton backends.
  ☆295Updated this week
• triton-inference-server / pytorch_backend
  The Triton backend for the PyTorch TorchScript models.
  ☆127Updated this week
• triton-inference-server / paddlepaddle_backend
  ☆32Updated 9 months ago
• inisis / OnnxSlim
  A Toolkit to Help Optimize Onnx Model
  ☆81Updated this week
• k9ele7en / Triton-TensorRT-Inference-CRAFT-pytorch
  Advanced inference pipeline using NVIDIA Triton Inference Server for CRAFT Text detection (Pytorch), included converter from Pytorch -> O…
  ☆32Updated 3 years ago
• NVIDIA / healthcare-on-tap-TRT-TRITON-demo
  Demonstration of the use of TensorRT and TRITON
  ☆17Updated 3 years ago
• Curt-Park / mnist-fastapi-celery-triton
  Simple example of FastAPI + Celery + Triton for benchmarking
  ☆61Updated 2 years ago
• onnx / neural-compressor
  Model compression for ONNX
  ☆75Updated this week
• triple-Mu / TensorRT2ONNX
  A tool convert TensorRT engine/plan to a fake onnx
  ☆37Updated 2 years ago
• triton-inference-server / model_analyzer
  Triton Model Analyzer is a CLI tool to help with better understanding of the compute and memory requirements of the Triton Inference Serv…
  ☆434Updated this week
• chiehpower / Setup-deeplearning-tools
  Set up CI in DL/ cuda/ cudnn/ TensorRT/ onnx2trt/ onnxruntime/ onnxsim/ Pytorch/ Triton-Inference-Server/ Bazel/ Tesseract/ PaddleOCR/ NV…
  ☆43Updated last year
• NVIDIA-AI-IOT / deepstream_libraries
  DeepStream Libraries offer CVCUDA, NvImageCodec, and PyNvVideoCodec modules as Python APIs for seamless integration into custom framewor…
  ☆25Updated last month
• levipereira / triton-server-yolo
  This repository serves as an example of deploying the YOLO models on Triton Server for performance and testing purposes
  ☆44Updated 5 months ago
• triton-inference-server / vllm_backend
  ☆192Updated this week
• PINTO0309 / scs4onnx
  A very simple tool that compresses the overall size of the ONNX model by aggregating duplicate constant values as much as possible.
  ☆52Updated 2 years ago
• triton-inference-server / common
  Common source, scripts and utilities shared across all Triton repositories.
  ☆62Updated this week
• NVIDIA-AI-IOT / deepstream_triton_model_deploy
  How to deploy open source models using DeepStream and Triton Inference Server
  ☆74Updated 4 months ago
• NVIDIA-AI-IOT / deepstream_parallel_inference_app
  A project demonstrating how to use nvmetamux to run multiple models in parallel.
  ☆95Updated last month
• qbxlvnf11 / convert-pytorch-onnx-tensorrt
  Converting weights of Pytorch models to ONNX & TensorRT engines
  ☆46Updated last year