dwarvesf / llm-hosting

This repository is designed for deploying and managing server processes that handle embeddings using the Infinity Embedding model or Large Language Models with an OpenAI compatible vLLM server.
20Updated 3 months ago

Alternatives and similar repositories for llm-hosting:

Users that are interested in llm-hosting are comparing it to the libraries listed below