SJTU-IPADS / PowerInfer

High-speed Large Language Model Serving for Local Deployment
8,161Updated last month

Alternatives and similar repositories for PowerInfer:

Users that are interested in PowerInfer are comparing it to the libraries listed below