infinigence / Semi-PDLinks

A prefill & decode disaggregated LLM serving framework with shared GPU memory and fine-grained compute isolation.
87Updated last month

Alternatives and similar repositories for Semi-PD

Users that are interested in Semi-PD are comparing it to the libraries listed below

Sorting: