morsoli / llmbenchmarkLinks
大模型API性能指标比较 - 深入分析TTFT、TPS等关键指标
☆19Updated last year
Alternatives and similar repositories for llmbenchmark
Users that are interested in llmbenchmark are comparing it to the libraries listed below
Sorting:
- NVIDIA TensorRT Hackathon 2023复赛选题：通义千问Qwen-7B用TensorRT-LLM模型搭建及优化☆43Updated 2 years ago
- 天池 NVIDIA TensorRT Hackathon 2023 —— 生成式AI模型优化赛 初赛第三名方案☆50Updated 2 years ago
- Bert TensorRT模 型加速部署☆10Updated 3 years ago
- 安卓手机部署DeepSeek-R1 蒸馏的1.5B模型☆22Updated 9 months ago
- YOLOv12 TensorRT 端到端模型加速推理和INT8量化实现☆13Updated 8 months ago
- deploy onnx models with TensorRT and LibTorch☆19Updated 4 years ago
- run ChatGLM2-6B in BM1684X☆50Updated last year
- 使用mnn-llm对GOT-OCR2.0进行推理☆15Updated last year
- Inference deployment of the llama3☆11Updated last year
- ☆27Updated 4 months ago
- Whisper in TensorRT-LLM☆16Updated 2 years ago
- ☢️ TensorRT 2023复赛——基于TensorRT-LLM的Llama模型推断加速优化☆50Updated 2 years ago
- ffmpeg+cuvid+tensorrt+multicamera☆12Updated 10 months ago
- 使用FastAPI+vLLM部署Qwen2.5☆24Updated last year
- paper-read-notes☆14Updated last year
- HunyuanDiT with TensorRT and libtorch☆18Updated last year
- ☆19Updated last year
- ☆27Updated last year
- 基于rknn的yolov5的cpp实现，包含各种依赖库，是一个完整工程，可直接编译运行☆21Updated 3 years ago
- 陆续开源医疗行业的深度学习模 型及数据集☆13Updated 3 years ago
- 大模型部署实战：TensorRT-LLM, Triton Inference Server, vLLM☆26Updated last year
- 国内外数据竞赛资讯整理☆18Updated 4 years ago
- 基于MNN-llm的安卓手机部署大语言模型：Qwen1.5-0.5B-Chat☆86Updated last year
- ggml学习笔记，ggml是一个机器学习的推理框架☆18Updated last year
- qwen2 and llama3 cpp implementation☆48Updated last year
- 使用ONNXRuntime部署Detic检测2万1千种类别的物体，包含C++和Python两个版本的程序☆17Updated 2 years ago
- 使用OpenCV部署L2CS-Net人脸朝向估计，包含C++和Python两个版本的程序，只依赖opencv库就可以运行☆20Updated 2 years ago
- A high-throughput and memory-efficient inference and serving engine for LLMs☆16Updated last year
- a simple lightweight large language model pipeline framework.☆28Updated 6 months ago
- 手摸手 美团 YOLOv6模型训练和TensorRT端到端部署方案教程☆34Updated 3 years ago