openai / tiktokenLinks

tiktoken is a fast BPE tokeniser for use with OpenAI's models.
16,097Updated last week

Alternatives and similar repositories for tiktoken

Users that are interested in tiktoken are comparing it to the libraries listed below

Sorting: