tensorchord / deepseek-api-arenaLinks
A benchmarking tool for comparing different LLM API providers' DeepSeek model deployments.
☆29Updated 5 months ago
Alternatives and similar repositories for deepseek-api-arena
Users that are interested in deepseek-api-arena are comparing it to the libraries listed below
Sorting:
- The driver for LMCache core to run in vLLM☆47Updated 6 months ago
- This repository contains statistics about the AI Infrastructure products.☆17Updated 6 months ago
- ☆31Updated 4 months ago
- CPM.cu is a lightweight, high-performance CUDA implementation for LLMs, optimized for end-device inference and featuring cutting-edge tec…☆175Updated this week
- OME is a Kubernetes operator for enterprise-grade management and serving of Large Language Models (LLMs)☆245Updated last week
- vLLM Router☆39Updated last year
- KV cache store for distributed LLM inference☆314Updated 2 months ago
- Make SGLang go brrr☆14Updated last week
- An Envoy inspired, ultimate LLM-first gateway for LLM serving and downstream application developers and enterprises☆23Updated 4 months ago
- Pretrain, finetune and serve LLMs on Intel platforms with Ray☆129Updated 2 weeks ago
- Common recipes to run vLLM☆119Updated this week
- Efficient, Flexible, and Highly Fault-Tolerant Model Service Management Based on SGLang☆56Updated 9 months ago
- Omni_Infer is a suite of inference accelerators designed for the Ascend NPU platform, offering native support and an expanding feature se…☆60Updated this week
- Genai-bench is a powerful benchmark tool designed for comprehensive token-level performance evaluation of large language model (LLM) serv…☆195Updated last week
- Fused Qwen3 MoE layer for faster training, compatible with HF Transformers, LoRA, 4-bit quant, Unsloth☆167Updated this week
- Turn PostgreSQL into your search engine in a Pythonic way.☆50Updated this week
- llm-inference is a platform for publishing and managing llm inference, providing a wide range of out-of-the-box features for model deploy…☆86Updated last year
- Inferflow is an efficient and highly configurable inference engine for large language models (LLMs).☆247Updated last year
- ☆55Updated 9 months ago
- ☆92Updated 5 months ago
- A prefill & decode disaggregated LLM serving framework with shared GPU memory and fine-grained compute isolation.☆105Updated 3 months ago
- Workflow Defined Engine☆24Updated 2 weeks ago
- MoonPalace(月宫)是由 Moonshot AI 月之暗面提供的 API 调试工具。☆203Updated 8 months ago
- Implemented a script that automatically adjusts Qwen3's inference and non-inference capabilities, based on an OpenAI-like API. The infere…☆22Updated 3 months ago
- GPUd automates monitoring, diagnostics, and issue identification for GPUs☆415Updated this week
- LLM Serving Performance Evaluation Harness☆79Updated 6 months ago
- Compare different hardware platforms via the Roofline Model for LLM inference tasks.☆112Updated last year
- Deploy ChatGLM on Modelz☆16Updated 2 years ago
- Skywork-MoE: A Deep Dive into Training Techniques for Mixture-of-Experts Language Models☆136Updated last year
- patches for huggingface transformers to save memory☆27Updated 2 months ago