tensorchord / deepseek-api-arenaLinks
A benchmarking tool for comparing different LLM API providers' DeepSeek model deployments.
☆29Updated 4 months ago
Alternatives and similar repositories for deepseek-api-arena
Users that are interested in deepseek-api-arena are comparing it to the libraries listed below
Sorting:
- The driver for LMCache core to run in vLLM☆45Updated 6 months ago
- ☆30Updated 3 months ago
- This repository contains statistics about the AI Infrastructure products.☆18Updated 5 months ago
- OME is a Kubernetes operator for enterprise-grade management and serving of Large Language Models (LLMs)☆209Updated this week
- vLLM Router☆39Updated last year
- CPM.cu is a lightweight, high-performance CUDA implementation for LLMs, optimized for end-device inference and featuring cutting-edge tec…☆169Updated last week
- Efficient, Flexible, and Highly Fault-Tolerant Model Service Management Based on SGLang☆55Updated 9 months ago
- Genai-bench is a powerful benchmark tool designed for comprehensive token-level performance evaluation of large language model (LLM) serv…☆185Updated this week
- Common recipes to run vLLM☆81Updated this week
- KV cache store for distributed LLM inference☆305Updated 2 months ago
- Deploy ChatGLM on Modelz☆15Updated 2 years ago
- Inferflow is an efficient and highly configurable inference engine for large language models (LLMs).☆246Updated last year
- A prefill & decode disaggregated LLM serving framework with shared GPU memory and fine-grained compute isolation.☆104Updated 2 months ago
- ☆92Updated 4 months ago
- Fused Qwen3 MoE layer for faster training, compatible with HF Transformers, LoRA, 4-bit quant, Unsloth☆142Updated last week
- Pretrain, finetune and serve LLMs on Intel platforms with Ray☆128Updated this week
- llm-inference is a platform for publishing and managing llm inference, providing a wide range of out-of-the-box features for model deploy…☆85Updated last year
- Turn PostgreSQL into your search engine in a Pythonic way.☆47Updated this week
- GPUd automates monitoring, diagnostics, and issue identification for GPUs☆409Updated this week
- ☆20Updated 3 months ago
- SJTU SE3357 操作系统笔记 OS Notes☆16Updated 2 years ago
- Implemented a script that automatically adjusts Qwen3's inference and non-inference capabilities, based on an OpenAI-like API. The infere…☆22Updated 3 months ago
- LLM Serving Performance Evaluation Harness☆79Updated 5 months ago
- Compare different hardware platforms via the Roofline Model for LLM inference tasks.☆110Updated last year
- MoonPalace(月宫)是由 Moonshot AI 月之暗面提供的 API 调试工具。☆203Updated 7 months ago
- An Envoy inspired, ultimate LLM-first gateway for LLM serving and downstream application developers and enterprises☆20Updated 3 months ago
- ☆55Updated 8 months ago
- vLLM performance dashboard☆33Updated last year
- Toolchain built around the Megatron-LM for Distributed Training☆50Updated last week
- ☆111Updated last month