EmbeddedLLM / embeddedllm

EmbeddedLLM: API server for Embedded Device Deployment. Currently support CUDA/OpenVINO/IpexLLM/DirectML/CPU
32Updated 4 months ago

Alternatives and similar repositories for embeddedllm:

Users that are interested in embeddedllm are comparing it to the libraries listed below