Yulv-git / Model-Inference-DeploymentLinks
A curated list of awesome inference deployment framework of artificial intelligence (AI) models. OpenVINO, TensorRT, MediaPipe, TensorFlow Lite, TensorFlow Serving, ONNX Runtime, LibTorch, NCNN, TNN, MNN, TVM, MACE, Paddle Lite, MegEngine Lite, OpenPPL, Bolt, ExecuTorch.
☆69Updated last year
Alternatives and similar repositories for Model-Inference-Deployment
Users that are interested in Model-Inference-Deployment are comparing it to the libraries listed below
Sorting:
- NVIDIA DLA-SW, the recipes and tools for running deep learning workloads on NVIDIA DLA cores for inference applications.☆218Updated last year
- ☆120Updated 2 years ago
- 该代码与B站上的视频 https://www.bilibili.com/video/BV18L41197Uz/?spm_id_from=333.788&vd_source=eefa4b6e337f16d87d87c2c357db8ca7 相关联。☆70Updated 2 years ago
- A simple tool that can generate TensorRT plugin code quickly.☆236Updated 2 years ago
- A large number of cuda/tensorrt cases . 大量案例来学习cuda/tensorrt☆157Updated 3 years ago
- YOLOv5 on Orin DLA☆215Updated last year
- An onnx-based quantitation tool.☆71Updated last year
- Serving Inside Pytorch☆163Updated last month
- Quick and Self-Contained TensorRT Custom Plugin Implementation and Integration☆71Updated 5 months ago
- ☆47Updated 2 years ago
- ☆140Updated last year
- TensorRT 2022 亚军方案,tensorrt加速mobilevit模型☆68Updated 3 years ago
- Offline Quantization Tools for Deploy.☆140Updated last year
- A Toolkit to Help Optimize Large Onnx Model☆161Updated last year
- A simple tutorial of SNPE.☆178Updated 2 years ago
- Using pattern matcher in onnx model to match and replace subgraphs.☆81Updated last year
- ☆43Updated 3 years ago
- ☆79Updated 2 years ago
- A tool convert TensorRT engine/plan to a fake onnx☆41Updated 2 years ago
- learning-cuda-trt☆118Updated 2 years ago
- ☆150Updated last year
- 高效部署:YOLO X, V3, V4, V5, V6, V7, V8, EdgeYOLO TRT推理 ™️ ,前后处理均由CUDA核函数实现 CPP/CUDA🚀☆50Updated 2 years ago
- NVIDIA TensorRT Hackathon 2023复赛选题:通义千问Qwen-7B用TensorRT-LLM模型搭建及优化☆43Updated 2 years ago
- Edgeai TIDL Tools and Examples - This repository contains Tools and example developed for Deep learning runtime (DLRT) offering provided …☆174Updated 3 weeks ago
- TensorRT 7 C++ (almost) minimal examples☆83Updated last year
- This is 8-bit quantization sample for yolov5. Both PTQ, QAT and Partial Quantization have been implemented, and present the results based…☆109Updated 3 years ago
- algorithm-cpp projects☆80Updated 3 years ago
- This is a repository to practice multi-thread programming in C++☆26Updated last year
- ☢️ TensorRT 2023复赛——基于TensorRT-LLM的Llama模型推断加速优化☆50Updated 2 years ago
- Simple demo of tensorrt plugin☆44Updated 4 years ago