microsoft / batch-inferenceLinks

Dynamic batching library for Deep Learning inference. Tutorials for LLM, GPT scenarios.
103Updated last year

Alternatives and similar repositories for batch-inference

Users that are interested in batch-inference are comparing it to the libraries listed below

Sorting: