openvinotoolkit / model_server
A scalable inference server for models optimized with OpenVINO™
☆675Updated this week
Related projects ⓘ
Alternatives and complementary repositories for model_server
- TensorFlow/TensorRT integration☆736Updated 11 months ago
- Inference Model Manager for Kubernetes☆46Updated 5 years ago
- Intel® AI Reference Models: contains Intel optimizations for running deep learning workloads on Intel® Xeon® Scalable processors and Inte…☆683Updated this week
- Triton Model Analyzer is a CLI tool to help with better understanding of the compute and memory requirements of the Triton Inference Serv…☆433Updated last week
- Run Generative AI models with simple C++/Python API and using OpenVINO Runtime☆152Updated this week
- This repository is a home to Intel® Deep Learning Streamer (Intel® DL Streamer) Pipeline Framework. Pipeline Framework is a streaming med…☆529Updated 3 weeks ago
- Neural Network Compression Framework for enhanced OpenVINO™ inference☆943Updated this week
- A multi-user, distributed computing environment for running DL model training experiments on Intel® Xeon® Scalable processor-based system…☆392Updated 6 months ago
- Actively maintained ONNX Optimizer☆647Updated 8 months ago
- Triton Python, C++ and Java client libraries, and GRPC-generated client examples for go, java and scala.☆570Updated this week
- 🤗 Optimum Intel: Accelerate inference with Intel optimization tools☆409Updated this week
- Train, Evaluate, Optimize, Deploy Computer Vision Models via OpenVINO™☆1,143Updated this week
- Explore the Capabilities of the TensorRT Platform☆260Updated 3 years ago
- The framework to generate a Dockerfile, build, test, and deploy a docker image with OpenVINO™ toolkit.☆59Updated last week
- ONNXMLTools enables conversion of models to ONNX☆1,024Updated 5 months ago
- Common utilities for ONNX converters☆251Updated 5 months ago
- A performant and modular runtime for TensorFlow☆756Updated last month
- OpenVINO operator for OpenShift and Kubernetes☆14Updated 2 months ago
- Common source, scripts and utilities for creating Triton backends.☆295Updated this week
- The Triton backend that allows running GPU-accelerated data pre-processing pipelines implemented in DALI's python API.☆125Updated 2 weeks ago
- Tensorflow Backend for ONNX☆1,284Updated 7 months ago
- Triton backend that enables pre-process, post-processing and other logic to be implemented in Python.☆553Updated this week
- ONNX-TensorRT: TensorRT backend for ONNX☆2,953Updated 2 weeks ago
- Samples for TensorRT/Deepstream for Tesla & Jetson☆1,141Updated last month
- Sample apps to demonstrate how to deploy models trained with TAO on DeepStream☆377Updated last month
- Triton Model Navigator is an inference toolkit designed for optimizing and deploying Deep Learning models with a focus on NVIDIA GPUs.☆185Updated 2 months ago
- Computation using data flow graphs for scalable machine learning☆67Updated this week
- This repository deploys YOLOv4 as an optimized TensorRT engine to Triton Inference Server☆279Updated 2 years ago
- Repository for OpenVINO's extra modules☆106Updated 2 weeks ago
- Save, Load Frozen Graph and Run Inference From Frozen Graph in TensorFlow 1.x and 2.x☆300Updated 3 years ago