tensorchord / deepseek-api-arenaLinks
A benchmarking tool for comparing different LLM API providers' DeepSeek model deployments.
☆29Updated 5 months ago
Alternatives and similar repositories for deepseek-api-arena
Users that are interested in deepseek-api-arena are comparing it to the libraries listed below
Sorting:
- The driver for LMCache core to run in vLLM☆50Updated 7 months ago
- This repository contains statistics about the AI Infrastructure products.☆17Updated 6 months ago
- ☆31Updated 5 months ago
- CPM.cu is a lightweight, high-performance CUDA implementation for LLMs, optimized for end-device inference and featuring cutting-edge tec…☆192Updated this week
- OME is a Kubernetes operator for enterprise-grade management and serving of Large Language Models (LLMs)☆273Updated this week
- Common recipes to run vLLM☆131Updated this week
- KV cache store for distributed LLM inference☆335Updated last week
- MoonPalace(月宫)是由 Moonshot AI 月之暗面提供的 API 调试工具。☆217Updated 8 months ago
- vLLM Router☆43Updated last year
- Make SGLang go brrr☆30Updated last week
- Pretrain, finetune and serve LLMs on Intel platforms with Ray☆132Updated 2 weeks ago
- Efficient, Flexible, and Highly Fault-Tolerant Model Service Management Based on SGLang☆56Updated 10 months ago
- Genai-bench is a powerful benchmark tool designed for comprehensive token-level performance evaluation of large language model (LLM) serv…☆211Updated 2 weeks ago
- Deploy ChatGLM on Modelz☆16Updated 2 years ago
- llm-inference is a platform for publishing and managing llm inference, providing a wide range of out-of-the-box features for model deploy…☆86Updated last year
- Omni_Infer is a suite of inference accelerators designed for the Ascend NPU platform, offering native support and an expanding feature se…☆71Updated last week
- ☆95Updated 5 months ago
- 国产加速卡-海光DCU实战(大模型训练、微调、推理 等)☆46Updated last month
- patches for huggingface transformers to save memory☆29Updated 3 months ago
- LLM Serving Performance Evaluation Harness☆79Updated 6 months ago
- ByteCheckpoint: An Unified Checkpointing Library for LFMs☆245Updated 2 months ago
- Fused Qwen3 MoE layer for faster training, compatible with HF Transformers, LoRA, 4-bit quant, Unsloth☆176Updated this week
- Compare different hardware platforms via the Roofline Model for LLM inference tasks.☆112Updated last year
- ☆26Updated 2 weeks ago
- GPUd automates monitoring, diagnostics, and issue identification for GPUs☆427Updated this week
- ☆131Updated 3 months ago
- A prefill & decode disaggregated LLM serving framework with shared GPU memory and fine-grained compute isolation.☆108Updated 4 months ago
- An Envoy inspired, ultimate LLM-first gateway for LLM serving and downstream application developers and enterprises☆23Updated 4 months ago
- ☆51Updated this week
- JAX backend for SGL☆60Updated this week