LLM 并发性能测试工具，支持自动化压力测试和性能报告生成。
☆268Dec 10, 2025Updated 7 months ago
Alternatives and similar repositories for llm-benchmark
Users that are interested in llm-benchmark are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- 安卓手机部署DeepSeek-R1 蒸馏的1.5B模型☆24Feb 4, 2025Updated last year
- A streamlined and customizable framework for efficient large model (LLM, VLM, AIGC) evaluation and performance benchmarking.☆3,144Updated this week
- HunyuanDiT with TensorRT and libtorch☆18May 22, 2024Updated 2 years ago
- 大模型推理压测☆50Jul 31, 2025Updated 11 months ago
- Community maintained hardware plugin for vLLM on Ascend☆2,488Updated this week
- 1-Click AI Models by DigitalOcean Gradient • AdDeploy popular AI models on DigitalOcean Gradient GPU virtual machines with just a single click. Zero configuration with optimized deployments.
- 大模型API性能指标比较 - 深入分析TTFT、TPS等关键指标☆20Sep 12, 2024Updated last year
- A prompt-aware LLM router that predicts which models can complete each request, then selects the cheapest capable one: 53.2% lower cost a…☆32Updated this week
- A GPU cluster manager for high-performance AI model serving (vLLM, SGLang) and on-demand SSH-accessible GPU instances.☆5,385Updated this week
- Dify-Plus 是 Dify 的企业级增强版，集成了基于 gin-vue-admin 的管理中心，并针对企业场景进行了功能优化。 🚀 Dify-Plus = 管理中心 + Dify 二开 。 特别说明： 本项目为开源社区的二次开发成果，严格遵循 Dify 原项目的版…☆2,206Updated this week
- YOLOv12 TensorRT 端到端模型加速推理和INT8量化实现☆14Mar 5, 2025Updated last year
- ☆13Feb 9, 2026Updated 5 months ago
- 大语言模型评估平台，支持多种评估基准、自定义数据集和性能测试。支持基于自定义数据集的RAG评估。☆97Aug 20, 2025Updated 11 months ago
- Whisper in TensorRT-LLM☆17Sep 21, 2023Updated 2 years ago
- 这是一个使用opencv读取视频并使用socket进行传输视频画面的脚本文件，相较于调用ffmpeg传输节约了90%的数据量☆11May 14, 2024Updated 2 years ago
- Managed Kubernetes at scale on DigitalOcean • AdDigitalOcean Kubernetes includes the control plane, bandwidth allowance, container registry, automatic updates, and more for free.
- Lightweight multi-agent orchestration engine☆15Jul 1, 2026Updated 3 weeks ago
- Flash Attention in ~100 lines of CUDA (forward pass only)☆12Jun 10, 2024Updated 2 years ago
- 一个轻量化的大模型推理框架☆23May 26, 2025Updated last year
- learn TensorRT from scratch🥰☆18Sep 29, 2024Updated last year
- 搜藏的希望的代码片段☆13Jun 6, 2023Updated 3 years ago
- Claude AI 的使用指南和最佳实践中文版 https://github.com/anthropics/claude-cookbooks☆17Feb 26, 2026Updated 5 months ago
- a ai infra framework for edge device base on nndeploy☆18Nov 27, 2025Updated 8 months ago
- Awesome code, projects, books, etc. related to CUDA☆38Jun 2, 2026Updated last month
- 使用mnn-llm对GOT-OCR2.0进行推理☆14Oct 2, 2024Updated last year
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- Swap GPT for any LLM by changing a single line of code. Xinference lets you run open-source, speech, and multimodal models on cloud, on-p…☆9,454Updated this week
- 高性能 高精度 大陆车牌、港澳车牌、台湾车牌 韩国车牌(South Korea LPR)识别 代码开源(ncnn移植）☆46Nov 5, 2025Updated 8 months ago
- Repository containing necessary files to run a server able to run Webots simulation☆12Apr 15, 2026Updated 3 months ago
- 基于chatglm3-6b生成ppt大纲以及ppt详情☆26Mar 10, 2024Updated 2 years ago
- 基于Dify自主创建的AI应用DSL工作流，你可以免费获取，无论是出于个人需求还是学习目的，它都能为您开启一段充满无限可能的智能之旅。☆2,618Jul 13, 2026Updated 2 weeks ago
- ☆15Dec 22, 2025Updated 7 months ago
- Open-source speech recognition toolkit for training, inference, streaming ASR, VAD, punctuation, speaker diarization pipelines, and OpenA…☆19,483Updated this week
- A high-throughput and memory-efficient inference and serving engine for LLMs☆87,210Updated this week
- 使用C++ tensorrt10 推理 ml-depth-pro 模型☆23Jan 8, 2025Updated last year
- GPU virtual machines on DigitalOcean Gradient AI • AdGet to production fast with high-performance AMD and NVIDIA GPUs you can spin up in seconds. The definition of operational simplicity.
- A2A Concept☆15Apr 10, 2025Updated last year
- Benchmarking the serving capabilities of vLLM☆59Aug 20, 2024Updated last year
- fastllm是后端无依赖的高性能大模型推理库。同时支持张量并行推理稠密模型和混合模式推理MOE模型，任意10G以上显卡即可推理满血DeepSeek。双路9004/9005服务器+单显卡部署DeepSeek满血满精度原版模型，单并发20tps；INT4量化模型单并发30tp…☆4,867Updated this week
- ☆31May 9, 2025Updated last year
- 2023 XFlops Training☆13Jan 23, 2024Updated 2 years ago
- AI-agent应用，基于GPT、langchain、function calling、Stable diffusion等的AI儿童绘本生成☆25Oct 11, 2023Updated 2 years ago
- Co-DETR (Detection Transformer) compiled from PyTorch to NVIDIA TensorRT☆20Apr 19, 2025Updated last year