LLM 并发性能测试工具，支持自动化压力测试和性能报告生成。
☆294Dec 10, 2025Updated 10 months ago
Alternatives and similar repositories for llm-benchmark
Users that are interested in llm-benchmark are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- 安卓手机部署DeepSeek-R1 蒸馏的1.5B模型☆26Feb 4, 2025Updated last year
- HunyuanDiT with TensorRT and libtorch☆18May 22, 2024Updated 2 years ago
- 大模型推理压测☆51Jul 31, 2025Updated last year
- Community maintained hardware plugin for vLLM on Huawei Ascend☆2,934Updated this week
- 大模型API性能指标比较 - 深入分析TTFT、TPS等关键指标☆20Sep 12, 2024Updated 2 years ago
- 1-Click AI Models by DigitalOcean Gradient • AdDeploy popular AI models on DigitalOcean Gradient GPU virtual machines with just a single click. Zero configuration with optimized deployments.
- 华为集合通信性能测试☆16May 27, 2024Updated 2 years ago
- A prompt-aware LLM router that predicts which models can complete each request, then selects the cheapest capable one: 53.2% lower cost a…☆40Sep 30, 2026Updated last week
- LLMPerf is a library for validating and benchmarking LLMs☆1,124Dec 9, 2024Updated last year
- opencv调用jetson/rk3588 mpp硬解码，重写了open与read函数，支持h264/h265☆14Aug 4, 2026Updated 2 months ago
- 基于Qwen2模型进行通用信息抽取【实体/关系/事件抽取】☆40Jul 10, 2024Updated 2 years ago
- YOLOv12 TensorRT 端到端模型加速推理和INT8量化实现☆14Mar 5, 2025Updated last year
- 大语言模型评估平台，支持多种评估基准、自定义数 据集和性能测试。支持基于自定义数据集的RAG评估。☆98Aug 20, 2025Updated last year
- 这是一个使用opencv读取视频并使用socket进行传输视频画面的脚本文件，相较于调用ffmpeg传输节约了90%的数据量☆11May 14, 2024Updated 2 years ago
- Lightweight multi-agent orchestration engine☆14Jul 1, 2026Updated 3 months ago
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- Flash Attention in ~100 lines of CUDA (forward pass only)☆12Jun 10, 2024Updated 2 years ago
- 一个轻量化的大模型推理框架☆23May 26, 2025Updated last year
- 搜藏的希望的代码片段☆13Jun 6, 2023Updated 3 years ago
- a ai infra framework for edge device base on nndeploy☆17Nov 27, 2025Updated 10 months ago
- Awesome code, projects, books, etc. related to CUDA☆45Aug 9, 2026Updated 2 months ago
- 使用mnn-llm对GOT-OCR2.0进行推理☆13Oct 2, 2024Updated 2 years ago
- Swap GPT for any LLM by changing a single line of code. Xinference lets you run open-source, speech, and multimodal models on cloud, on-p…☆9,606Updated this week
- Repository containing necessary files to run a server able to run Webots simulation☆12Apr 15, 2026Updated 5 months ago
- ☆25Mar 31, 2025Updated last year
- 1-Click AI Models by DigitalOcean Gradient • AdDeploy popular AI models on DigitalOcean Gradient GPU virtual machines with just a single click. Zero configuration with optimized deployments.
- 基于chatglm3-6b生成ppt大纲以及ppt详情☆26Mar 10, 2024Updated 2 years ago
- 基于Dify自主创建的AI应用DSL工作流，你可以免费获取，无论是出于个人需求还是学习目的，它都能为您开启一段充满无限可能的智能 之旅。☆2,664Jul 13, 2026Updated 2 months ago
- To convert CosyVoice model to ONNX☆17Dec 22, 2025Updated 9 months ago
- Open-source speech recognition toolkit for training, inference, streaming ASR, VAD, punctuation, speaker diarization pipelines, and OpenA…☆20,622Updated this week
- TCM Lingdan LLM☆54Aug 15, 2026Updated last month
- 使用C++ tensorrt10 推理 ml-depth-pro 模型☆23Jan 8, 2025Updated last year
- A2A Concept☆15Apr 10, 2025Updated last year
- Benchmarking the serving capabilities of vLLM☆59Aug 20, 2024Updated 2 years ago
- fastllm是后端无依赖的高性能大模型推理库。同时支持张量并行推理稠密模型和混合模式推理MOE模型，任意10G以上显卡即可推理满血DeepSeek。双路9004/9005服务器+单显卡部署DeepSeek满血满精度原版模型，单并发20tps；INT4量化模型单并发30tp…☆5,100Updated this week
- AI Agents on DigitalOcean Gradient AI Platform • AdBuild production-ready AI agents using customizable tools or access multiple LLMs through a single endpoint. Create custom knowledge bases or connect external data.
- AI-agent应用，基于GPT、langchain、function calling、Stable diffusion等的AI儿童绘本生成☆25Oct 11, 2023Updated 2 years ago
- Co-DETR (Detection Transformer) compiled from PyTorch to NVIDIA TensorRT☆21Apr 19, 2025Updated last year
- This is a repository to practice multi-thread programming in C++☆33Feb 21, 2024Updated 2 years ago
- LMCache-Ascend is a plugin for running LMCache on the Ascend NPU.☆90Sep 29, 2026Updated last week
- ToyNLP: Learning NLP from Scratch☆32Jun 19, 2026Updated 3 months ago
- Auto Thinking Mode switch for Qwen3 in Open webui☆70May 8, 2025Updated last year
- A simple neural network inference framework☆25Aug 1, 2023Updated 3 years ago