yanxiyue / vllm

A high-throughput and memory-efficient inference and serving engine for LLMs
17Updated last year

Alternatives and similar repositories for vllm:

Users that are interested in vllm are comparing it to the libraries listed below