thansen0 / fastllm.cpp

A low latency, fault tolerant API for accessing LLM's written in C++ using llama.cpp.
9Updated last week

Alternatives and similar repositories for fastllm.cpp:

Users that are interested in fastllm.cpp are comparing it to the libraries listed below