bentoml / OpenLLM

Run any open-source LLMs, such as DeepSeek and Llama, as OpenAI compatible API endpoint in the cloud.
10,566Updated this week

Alternatives and similar repositories for OpenLLM:

Users that are interested in OpenLLM are comparing it to the libraries listed below