llm-db / FineInferLinks

Deferred Continuous Batching in Resource-Efficient Large Language Model Serving (EuroMLSys 2024)
16Updated last year

Alternatives and similar repositories for FineInfer

Users that are interested in FineInfer are comparing it to the libraries listed below

Sorting: