Equationliu / Kangaroo

[NeurIPS 2024] The official implementation of "Kangaroo: Lossless Self-Speculative Decoding for Accelerating LLMs via Double Early Exiting"
51Updated 8 months ago

Alternatives and similar repositories for Kangaroo:

Users that are interested in Kangaroo are comparing it to the libraries listed below