Infini-AI-Lab / MagicDec

[ICLR2025] Breaking Throughput-Latency Trade-off for Long Sequences with Speculative Decoding
109Updated 3 months ago

Alternatives and similar repositories for MagicDec:

Users that are interested in MagicDec are comparing it to the libraries listed below