niieani / gpt-tokenizerLinks

The fastest JavaScript BPE Tokenizer Encoder Decoder for OpenAI's GPT models (o1, o3, o4, gpt-4o, gpt-4, etc.). Port of OpenAI's tiktoken with additional features.
578Updated last week

Alternatives and similar repositories for gpt-tokenizer

Users that are interested in gpt-tokenizer are comparing it to the libraries listed below

Sorting: