morsoli / llmbenchmarkLinks
大模型API性能指标比较 - 深入分析TTFT、TPS等关键指标
☆19Updated last year
Alternatives and similar repositories for llmbenchmark
Users that are interested in llmbenchmark are comparing it to the libraries listed below
Sorting:
- 安卓手机部署DeepSeek-R1 蒸馏的1.5B模型☆22Updated 8 months ago
- Bert TensorRT模型加速部署☆10Updated 3 years ago
- NVIDIA TensorRT Hackathon 2023复赛选题：通义千问Qwen-7B用TensorRT-LLM模型搭建及优化☆42Updated last year
- 天池 NVIDIA TensorRT Hackathon 2023 —— 生成式AI模型优化赛 初赛第三名方案☆50Updated 2 years ago
- YOLOv12 TensorRT 端到端模型加速推理和INT8量化实现☆13Updated 7 months ago
- deploy onnx models with TensorRT and LibTorch☆19Updated 3 years ago
- run ChatGLM2-6B in BM1684X☆49Updated last year
- Whisper in TensorRT-LLM☆16Updated 2 years ago
- ☆27Updated 3 months ago
- 使用mnn-llm对GOT-OCR2.0进行推理☆15Updated last year
- a simple lightweight large language model pipeline framework.☆26Updated 5 months ago
- ffmpeg+cuvid+tensorrt+multicamera☆12Updated 9 months ago
- Inference deployment of the llama3☆11Updated last year
- paper-read-notes☆12Updated last year
- 使用FastAPI+vLLM部署Qwen2.5☆23Updated last year
- 基于rknn的yolov5的cpp实现，包含各种依赖库，是一个完整工程，可直接编译运行☆21Updated 3 years ago
- Python scripts performing Open Vocabulary Object Detection using the YOLO-World model in ONNX.☆60Updated last year
- 手摸手 美团 YOLOv6模型训练和TensorRT端到端部署方案教程☆33Updated 3 years ago
- 大模型驱动的虚拟主播☆12Updated last year
- qwen2 and llama3 cpp implementation☆47Updated last year
- 这是一个使用opencv读取视频并使用socket进行传输视频画面的脚本文件，相较于调用ffmpeg传输节约了90%的数据量☆11Updated last year
- HunyuanDiT with TensorRT and libtorch☆18Updated last year
- 国内外数据竞赛资讯整理☆18Updated 3 years ago
- ☢️ TensorRT 2023复赛——基于TensorRT-LLM的Llama模型推断加速优化☆50Updated last year
- ☆10Updated last year
- 大模型部署实战：TensorRT-LLM, Triton Inference Server, vLLM☆26Updated last year
- FastSAM 部署版本，便于移植不同平，部署简单、运行速度快。☆23Updated last year
- 使用OpenCV+onnxruntime部署开放域目标检测，包含C++和Python两个版本的程序☆11Updated last year
- ☆18Updated last year
- 使用ONNXRuntime部署Detic检测2万1千种类别的物体，包含C++和Python两个版本的程序☆17Updated 2 years ago