LLM 并发性能测试工具，支持自动化压力测试和性能报告生成。
☆277Dec 10, 2025Updated 8 months ago
Alternatives and similar repositories for llm-benchmark
Users that are interested in llm-benchmark are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- 安卓手机部署DeepSeek-R1 蒸馏的1.5B模型☆25Feb 4, 2025Updated last year
- HunyuanDiT with TensorRT and libtorch☆18May 22, 2024Updated 2 years ago
- Community maintained hardware plugin for vLLM on Ascend☆2,760Updated this week
- 大模型API性能指标比较 - 深入分析TTFT、TPS等关键指标☆20Sep 12, 2024Updated last year
- 华为集合通信性能测试☆16May 27, 2024Updated 2 years ago
- GPU virtual machines on DigitalOcean Gradient AI • AdGet to production fast with high-performance AMD and NVIDIA GPUs you can spin up in seconds. The definition of operational simplicity.
- A prompt-aware LLM router that predicts which models can complete each request, then selects the cheapest capable one: 53.2% lower cost a…☆39Updated this week
- opencv调用jetson/rk3588 mpp硬解码，重写了open与read函数，支持h264/h265☆14Aug 4, 2026Updated last month
- A GPU cluster manager for high-performance AI model serving (vLLM, SGLang) and on-demand SSH-accessible GPU instances.☆5,609Updated this week
- Dify-Plus 是 Dify 的企业级增强版，集成了基于 gin-vue-admin 的管理中心，并针对企业场景进行了功能优化。 🚀 Dify-Plus = 管理中心 + Dify 二开 。 特别说明： 本项目为开源社区的二次开发成果，严格遵循 Dify 原项目的版…☆2,241Jul 21, 2026Updated last month
- 基于Qwen2模型进行通用信息抽取【实体/关系/事件抽取】☆41Jul 10, 2024Updated 2 years ago
- YOLOv12 TensorRT 端到端模型加速推理和INT8量化实现☆14Mar 5, 2025Updated last year
- ☆13Feb 9, 2026Updated 6 months ago
- ☆12Sep 10, 2017Updated 8 years ago
- 大语言模型评估平台，支持多种评估基准、自定义数据集和性能测试。支持基于自定义数据集的RAG评估。☆98Aug 20, 2025Updated last year
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- 这是一个使用opencv读取视频并使用socket进行传输视频画面的脚本文件，相较于调用ffmpeg传输节约了90%的数据量☆11May 14, 2024Updated 2 years ago
- Lightweight multi-agent orchestration engine☆14Jul 1, 2026Updated 2 months ago
- 这是一个用于 Linux 服务器的自动化安全合规检测脚本，遵循中国《GB/T 22239-2019 信息安全技术 网络安全等级保护基本要求》（等保三级）标准。脚本采用纯检测模式，不会修改系统，生成详细的 HTML 报告，覆盖身份鉴别、访问控制、安全审计、入侵防范和网络安全等…☆29Sep 4, 2025Updated last year
- 一个轻量化的大模型推理框架☆23May 26, 2025Updated last year
- learn TensorRT from scratch🥰☆18Sep 29, 2024Updated last year
- 搜藏的希望的代码片段☆13Jun 6, 2023Updated 3 years ago
- a ai infra framework for edge device base on nndeploy☆18Nov 27, 2025Updated 9 months ago
- Awesome code, projects, books, etc. related to CUDA☆41Aug 9, 2026Updated 3 weeks ago
- 使用mnn-llm对GOT-OCR2.0进行推理☆13Oct 2, 2024Updated last year
- Virtual machines for every use case on DigitalOcean • AdGet dependable uptime with 99.99% SLA, simple security tools, and predictable monthly pricing with DigitalOcean's virtual machines, called Droplets.
- Fork Nacos-Plugin, handle multiple datasource☆14Mar 22, 2026Updated 5 months ago
- ☆17Nov 3, 2013Updated 12 years ago
- Swap GPT for any LLM by changing a single line of code. Xinference lets you run open-source, speech, and multimodal models on cloud, on-p…☆9,544Updated this week
- 高性能 高精度 大陆车牌、港澳车牌、台湾车牌 韩国车牌(South Korea LPR)识别 代码开源(ncnn移植）☆49Nov 5, 2025Updated 10 months ago
- 基于chatglm3-6b生成ppt大纲以及ppt详情☆26Mar 10, 2024Updated 2 years ago
- 基于Dify自主创建的AI应用DSL工作流，你可以免费获取，无论是出于个人需求还是学习目的，它都能为您开启一段充满无限可能的智能之旅。☆2,649Jul 13, 2026Updated last month
- To convert CosyVoice model to ONNX☆16Dec 22, 2025Updated 8 months ago
- Open-source speech recognition toolkit for training, inference, streaming ASR, VAD, punctuation, speaker diarization pipelines, and OpenA…☆20,183Updated this week
- A high-throughput and memory-efficient inference and serving engine for LLMs☆91,027Updated this week
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- 使用C++ tensorrt10 推理 ml-depth-pro 模型☆23Jan 8, 2025Updated last year
- A2A Concept☆15Apr 10, 2025Updated last year
- Benchmarking the serving capabilities of vLLM☆59Aug 20, 2024Updated 2 years ago
- fastllm是后端无依赖的高性能大模型推理库。同时支持张量并行推理稠密模型和混合模式推理MOE模型，任意10G以上显卡即可推理满血DeepSeek。双路9004/9005服务器+单显卡部署DeepSeek满血满精度原版模型，单并发20tps；INT4量化模型单并发30tp…☆4,954Updated this week
- ☆31May 9, 2025Updated last year
- This is a repository to practice multi-thread programming in C++☆32Feb 21, 2024Updated 2 years ago
- LMCache-Ascend is a plugin for running LMCache on the Ascend NPU.☆87Updated this week