Efficient, Flexible, and Highly Fault-Tolerant Model Service Management Based on SGLang
☆63Nov 8, 2024Updated last year
Alternatives and similar repositories for ModelServer
Users that are interested in ModelServer are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- ☆13Dec 9, 2024Updated last year
- CUDA Graph runtime tensor debugging utilities for PyTorch☆18Jul 12, 2026Updated 3 months ago
- An auxiliary project analysis of the characteristics of KV in DiT Attention.☆34Nov 29, 2024Updated last year
- ☆12May 26, 2026Updated 5 months ago
- Quantized Attention on GPU☆45Nov 22, 2024Updated last year
- Managed Database hosting by DigitalOcean • AdPostgreSQL, MySQL, MongoDB, Kafka, Valkey, and OpenSearch available. Automatically scale up storage and focus on building your apps.
- RISCV C and Triton AI-Benchmark☆26Jan 28, 2026Updated 9 months ago
- Lightweight Benchmark Server for Agentic GPU Programming☆58Oct 22, 2026Updated last week
- Running inference on the ZeroSCROLLS benchmark☆22Apr 18, 2024Updated 2 years ago
- DeepSeek-V3/R1 inference performance simulator☆199Mar 27, 2025Updated last year
- Materials for learning SGLang☆897Jan 5, 2026Updated 9 months ago
- ☆156Mar 4, 2025Updated last year
- ☆13Jan 6, 2026Updated 9 months ago
- ☆17Mar 24, 2026Updated 7 months ago
- ☆151Jan 9, 2025Updated last year
- End-to-end encrypted email - Proton Mail • AdSpecial offer: 40% Off Yearly / 80% Off First Month. All Proton services are open source and independently audited for security.
- Beyond KV Caching: Shared Attention for Efficient LLMs☆20Jul 19, 2024Updated 2 years ago
- ☆99Apr 2, 2025Updated last year
- ggml学习笔记，ggml是一个机器学习的推理框架☆18Mar 24, 2024Updated 2 years ago
- Pytorch process group third-party plugin for UCC☆22Apr 15, 2024Updated 2 years ago
- FlowWalker is a practical off-line taint analysis framework with high efficiency and low overhead.☆12Feb 11, 2015Updated 11 years ago
- A single-file educational implementation for understanding vLLM's core concepts and running LLM inference.☆47Apr 7, 2026Updated 6 months ago
- Mix kubebuilder and code-generator example☆22Jul 3, 2020Updated 6 years ago
- fastDNS is an authoritative only, high performance, simple and open source name server based on DPDK and NSD server☆17Nov 13, 2017Updated 8 years ago
- Papers relevant with graph and GPU☆12Jan 1, 2026Updated 10 months ago
- Managed Database hosting by DigitalOcean • AdPostgreSQL, MySQL, MongoDB, Kafka, Valkey, and OpenSearch available. Automatically scale up storage and focus on building your apps.
- A high-performance, concurrent hash table☆24Dec 15, 2013Updated 12 years ago
- Learn about PTX instructions ldmatrix and stmatrix☆12Sep 5, 2025Updated last year
- ☆23Jul 13, 2026Updated 3 months ago
- 使用 cutlass 仓库在 ada 架构上实现 fp8 的 flash attention☆82Aug 12, 2024Updated 2 years ago
- Memory footprint reduction for transformer models☆11Jan 24, 2023Updated 3 years ago
- ☆553Aug 27, 2026Updated 2 months ago
- ReasoningLens: a user-friendly toolkit to visualize, understand, and debug model reasoning chains.☆27Jul 7, 2026Updated 3 months ago
- ☆54Aug 6, 2024Updated 2 years ago
- 登录脚本☆12Nov 4, 2022Updated 3 years ago
- Bare Metal GPUs on DigitalOcean Gradient AI • AdPurpose-built for serious AI teams training foundational models, running large-scale inference, and pushing the boundaries of what's possible.
- My learning notes for ML SYS.☆7,628Oct 21, 2026Updated last week
- Using PDFPlumber for PDF data extraction☆15May 31, 2017Updated 9 years ago
- 使用Qwen3-ASR模型生成电影字幕☆13Jan 31, 2026Updated 9 months ago
- ☆107Sep 9, 2024Updated 2 years ago
- FaceDetect is one of the purpose built models from NVIDIA GPU Cloud (NGC). In this project, we demonstrate how it can be deployed and uti…☆11Apr 15, 2021Updated 5 years ago
- All reduce implementation and analysis of BDE vs Ring primitives☆11Sep 7, 2022Updated 4 years ago
- Course project for CS410. Drug Molecular Toxicity Prediction with GCN + Cloud ML Infra.☆10Apr 6, 2021Updated 5 years ago