LLM 并发性能测试工具，支持自动化压力测试和性能报告生成。
☆237Dec 10, 2025Updated 4 months ago
Alternatives and similar repositories for llm-benchmark
Users that are interested in llm-benchmark are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- 安卓手机部署DeepSeek-R1 蒸馏的1.5B模型☆24Feb 4, 2025Updated last year
- 大模型API性能指标比较 - 深入分析TTFT、TPS等关键指标☆20Sep 12, 2024Updated last year
- opencv调用jetson/rk3588 mpp硬解码，重写了open与read函数，支持h264/h265☆14Nov 27, 2025Updated 4 months ago
- ☆20Dec 29, 2023Updated 2 years ago
- YOLOv12 TensorRT 端到端模型加速推理和INT8量化实现☆13Mar 5, 2025Updated last year
- Deploy open-source AI quickly and easily - Bonus Offer • AdRunpod Hub is built for open source. One-click deployment and autoscaling endpoints without provisioning your own infrastructure.
- 大模型推理压测☆47Jul 31, 2025Updated 8 months ago
- Community maintained hardware plugin for vLLM on Ascend☆1,937Updated this week
- ☆14Feb 9, 2026Updated 2 months ago
- A streamlined and customizable framework for efficient large model (LLM, VLM, AIGC) evaluation and performance benchmarking.☆2,685Updated this week
- 这是一个使用opencv读取视频并使用socket进行传输视频画面的脚本文件，相较于调用ffmpeg传输节约了90%的数据量☆11May 14, 2024Updated last year
- Whisper in TensorRT-LLM☆17Sep 21, 2023Updated 2 years ago
- learn TensorRT from scratch🥰☆18Sep 29, 2024Updated last year
- Dify-Plus 是 Dify 的企业级增强版，集成了基于 gin-vue-admin 的管理中心，并针对企业场景进行了功能优化。 🚀 Dify-Plus = 管理中心 + Dify 二开 。 特别说明： 本项目为开源社区的二次开发成果，严格遵循 Dify 原项目的版…☆2,121Updated this week
- Co-DETR (Detection Transformer) compiled from PyTorch to NVIDIA TensorRT☆20Apr 19, 2025Updated last year
- Managed Kubernetes at scale on DigitalOcean • AdDigitalOcean Kubernetes includes the control plane, bandwidth allowance, container registry, automatic updates, and more for free.
- 使用mnn-llm对GOT-OCR2.0进行推理☆14Oct 2, 2024Updated last year
- 高性能 高精度 大陆车牌、港澳车牌、台湾车牌 韩国车牌(South Korea LPR)识别 代码开源(ncnn移植）☆43Nov 5, 2025Updated 5 months ago
- A GPU cluster manager that configures and orchestrates inference engines like vLLM and SGLang for high-performance AI model deployment.☆4,832Updated this week
- SuperPoint and LightGlue with TensorRT. Deploy with C++.☆29Dec 14, 2023Updated 2 years ago
- Awesome code, projects, books, etc. related to CUDA☆32Mar 30, 2026Updated 2 weeks ago
- [2025AIAgent / 2025InternLab]An agent that provides free and flexible access to Search external knowledge.☆23Feb 18, 2026Updated 2 months ago
- A simple neural network inference framework☆25Aug 1, 2023Updated 2 years ago
- a simple lightweight large language model pipeline framework.☆28Apr 25, 2025Updated 11 months ago
- 学习vLLM，使用vLLM部署Qwen2-0.5B的模型，并使用docker部署。☆20Jun 22, 2024Updated last year
- Simple, predictable pricing with DigitalOcean hosting • AdAlways know what you'll pay with monthly caps and flat pricing. Enterprise-grade infrastructure trusted by 600k+ customers.
- 一个基于FastAPI和React的智能体系统，支持多智能体管理、mcp管理、知识库、聊天对话等功能。An intelligent agent system based on FastAPI and React, supporting multi-agent managem…☆24Jan 25, 2026Updated 2 months ago
- 用于SenseVoice的api项目，输出带时间戳字幕☆43Oct 28, 2024Updated last year
- Fully ready to use out of the box Docker Compose | 充分开箱即用的 Docker Compose☆24Updated this week
- [DAC'25] Official implement of "HybriMoE: Hybrid CPU-GPU Scheduling and Cache Management for Efficient MoE Inference"☆110Dec 15, 2025Updated 4 months ago
- FastAPI Chat App with WebSockets is an open-source real-time chat application built on the FastAPI framework. This project provides a rob…☆12Jan 7, 2024Updated 2 years ago
- 使用pytorch_quantization对yolov8进行量化☆118Nov 10, 2023Updated 2 years ago
- Flutter embedder for Tizen☆13Apr 7, 2026Updated last week
- MinerU免安装部署一键启动整合包☆18Oct 24, 2025Updated 5 months ago
- 基于SenseVoice的funasr版本进行的api发布，可以无缝对接oneapi☆93Sep 5, 2024Updated last year
- 1-Click AI Models by DigitalOcean Gradient • AdDeploy popular AI models on DigitalOcean Gradient GPU virtual machines with just a single click. Zero configuration with optimized deployments.
- DETR tensor去除推理过程无用辅助头+fp16部署再次加速+解决转tensorrt 输出全为0问题的新方法。☆11Jan 9, 2024Updated 2 years ago
- ☆34Mar 27, 2026Updated 3 weeks ago
- BiLSTM-CRF-NER☆34Apr 7, 2022Updated 4 years ago
- You say it. AutoCode builds it. 38 professional skills, persistent memory, 60%+ dev cost savings. Zero dependencies. Free forever.☆69Mar 20, 2026Updated 3 weeks ago
- fast-embeddings-api☆16Nov 23, 2023Updated 2 years ago
- ☆32Nov 19, 2025Updated 5 months ago
- APEX+ is an LLM Serving Simulator☆44Jun 16, 2025Updated 10 months ago