microsoft / sarathi-serveLinks

A low-latency & high-throughput serving engine for LLMs
370Updated this week

Alternatives and similar repositories for sarathi-serve

Users that are interested in sarathi-serve are comparing it to the libraries listed below

Sorting: