Alternatives and similar repositories for Model-Inference-Deployment:

Users that are interested in Model-Inference-Deployment are comparing it to the libraries listed below

• Oldpan / DeployIsAllYouNeed
  ☆120Updated last year
• NVIDIA / Deep-Learning-Accelerator-SW
  NVIDIA DLA-SW, the recipes and tools for running deep learning workloads on NVIDIA DLA cores for inference applications.
  ☆193Updated 10 months ago
• TRT2022 / trtllm-llama
  ☢️ TensorRT 2023复赛——基于TensorRT-LLM的Llama模型推断加速优化
  ☆46Updated last year
• ModelTC / Dipoorlet
  Offline Quantization Tools for Deploy.
  ☆127Updated last year
• FeiGeChuanShu / trt2023
  NVIDIA TensorRT Hackathon 2023复赛选题：通义千问Qwen-7B用TensorRT-LLM模型搭建及优化
  ☆42Updated last year
• harleyszhang / lite_llama
  A light llama-like llm inference framework based on the triton kernel.
  ☆108Updated last week
• sesmfs / onnx_quant_tool
  An onnx-based quantitation tool.
  ☆71Updated last year
• torchpipe / torchpipe
  Serving Inside Pytorch
  ☆160Updated this week
• lxl24 / SwinTransformerV2_TensorRT
  For 2022 Nvidia Hackathon
  ☆20Updated 2 years ago
• NVIDIA-AI-IOT / tensorrt_plugin_generator
  A simple tool that can generate TensorRT plugin code quickly.
  ☆230Updated last year
• zjhellofss / KuiperCourse
  b站上的课程
  ☆74Updated last year
• atanmarko / ncnn-with-cuda
  Tencent NCNN with added CUDA support
  ☆69Updated 4 years ago
• ArtyZe / yolo_quantization
  Based of paper "Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference"
  ☆63Updated 4 years ago
• maggiez0138 / yolov5_quant_sample
  This is 8-bit quantization sample for yolov5. Both PTQ, QAT and Partial Quantization have been implemented, and present the results based…
  ☆102Updated 2 years ago
• leimao / TensorRT-Custom-Plugin-Example
  Quick and Self-Contained TensorRT Custom Plugin Implementation and Integration
  ☆57Updated 10 months ago
• triple-Mu / TensorRT2ONNX
  A tool convert TensorRT engine/plan to a fake onnx
  ☆38Updated 2 years ago
• HeKun-NVIDIA / AI-Blog
  Collection of blogs on AI development
  ☆19Updated 5 months ago
• BBuf / onnx_learn
  ☆99Updated 3 years ago
• MegEngine / mgeconvert
  MegEngine到其他框架的转换器
  ☆69Updated last year
• tsingmicro-toolchain / OnnxSlim
  A Toolkit to Help Optimize Large Onnx Model
  ☆153Updated 11 months ago
• kalfazed / multi-thread-programming
  This is a repository to practice multi-thread programming in C++
  ☆24Updated last year
• TRT2022 / ControlNet_TensorRT
  天池 NVIDIA TensorRT Hackathon 2023 —— 生成式AI模型优化赛 初赛第三名方案
  ☆49Updated last year
• BBuf / ArmNeonOptimization
  arm-neon
  ☆90Updated 8 months ago
• NVIDIA-AI-IOT / cuDLA-samples
  YOLOv5 on Orin DLA
  ☆198Updated last year
• BBuf / giantpandacv.com
  www.giantpandacv.com
  ☆148Updated 10 months ago
• basaltzhang / tensorrt-plugin-demo
  Simple demo of tensorrt plugin
  ☆44Updated 3 years ago
• jinmin527 / learning-cuda-trt
  A large number of cuda/tensorrt cases . 大量案例来学习cuda/tensorrt
  ☆131Updated 2 years ago
• Wulingtian / nanodet_tensorrt_int8
  nanodet int8 量化，实测推理2ms一帧！
  ☆37Updated 4 years ago
• DataXujing / lanenet-tensorrt
  车道线检测Lanenet TensorRT加速C++实现
  ☆21Updated 3 years ago
• bindog / onnx-surgery
  ☆80Updated 4 years ago