facebookresearch / Exact-Byte-Level-Probabilities-from-Tokenized-LMsView on GitHub
Example implementation of "Exact Byte-Level Probabilities from Tokenized Language Models for FIM-Tasks and Model Ensembles" by Buu Phan, Brandon Amos, Itai Gat, Marton Havasi, Matthew Muckley, and Karen UllrichWork conducted as part of a MetaAI internship.
18Jan 22, 2026Updated 2 months ago

Alternatives and similar repositories for Exact-Byte-Level-Probabilities-from-Tokenized-LMs

Users that are interested in Exact-Byte-Level-Probabilities-from-Tokenized-LMs are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.

Sorting:

Are these results useful?