Yoosu-L / llmapibenchmarkLinks
The LLM API Benchmark Tool is a flexible Go-based utility designed to measure and analyze the performance of OpenAI-compatible API endpoints across different concurrency levels.
☆25Updated 3 months ago
Alternatives and similar repositories for llmapibenchmark
Users that are interested in llmapibenchmark are comparing it to the libraries listed below
Sorting:
- LM inference server implementation based on *.cpp.☆203Updated this week
- Open Source Text Embedding Models with OpenAI Compatible API☆153Updated 10 months ago
- A text-to-speech and speech-to-text server compatible with the OpenAI API, supporting Whisper, FunASR, Bark, and CosyVoice backends.☆118Updated this week
- The latest graphrag interface is used, using the local ollama to provide the LLM interface.Support for using the pip installation☆150Updated 7 months ago
- Library for model distillation☆142Updated 3 months ago
- Using GPT to parse PDF☆98Updated 9 months ago
- DashInfer is a native LLM inference engine aiming to deliver industry-leading performance atop various hardware architectures, including …☆254Updated this week
- LLM Inference benchmark☆419Updated 10 months ago
- bisheng-unstructured library☆48Updated 2 weeks ago
- Clone of https://r.jina.ai which is deployable locally☆44Updated 8 months ago
- gpt_server是一个用于生产级部署LLMs、Embedding、Reranker、ASR和TTS的开源框架。☆184Updated last week
- 通过该项目将Dify通过Pipeline接入OpenwebUI,可以兼并OpenwebUI的前端优势和相应生态以及Dify强大的模型可拓展性和Workflow的效益。☆32Updated 6 months ago
- 基于 Dify + Langfuse 的自动化评估服务☆63Updated last week
- A high-throughput and memory-efficient inference and serving engine for LLMs☆130Updated 11 months ago
- Comparison of Language Model Inference Engines☆217Updated 5 months ago
- run DeepSeek-R1 GGUFs on KTransformers☆231Updated 3 months ago
- ☆116Updated last month
- Awesome Code Action - DeepWebSearch AgentKit App. Build with 🤗 Hugging Face smolagents framework☆40Updated this week
- Production ready LLM model compression/quantization toolkit with hw accelerated inference support for both cpu/gpu via HF, vLLM, and SGLa…☆590Updated last week
- The Level-Navi Agent, a framework that requires no training and utilizes large language models for deep query understanding and precise s…☆79Updated 5 months ago
- Multi-Faceted AI Agent and Workflow Autotuning. Automatically optimizes LangChain, LangGraph, DSPy programs for better quality, lower exe…☆236Updated 3 weeks ago
- GLM-4 series: Open Multilingual Multimodal Chat LMs | 开源多语言多模态对话模型☆26Updated last month
- AI for all: Build the large graph of the language models☆266Updated last year
- Pretrain, finetune and serve LLMs on Intel platforms with Ray☆127Updated last month
- [ACL2025 demo track] ROGRAG: A Robustly Optimized GraphRAG Framework☆137Updated this week
- 与 https://github.com/tonori/mem0ai-api 配合使用的非官方的 mem0ai provider.☆48Updated 10 months ago
- MCP Agent Graph (MAG) is an agent development framework for rapidly building agent systems. This project is based on graphs, nodes, and M…☆46Updated this week
- An enterprise-grade AI retriever designed to streamline AI integration into your applications, ensuring cutting-edge accuracy.☆284Updated last month
- Jina DeepSearch UI☆110Updated 3 weeks ago
- Multi-Agents & Plugins repo for DB-GPT, Can complete various tasks around databases.☆101Updated last year