kaisugi / gpt4_vocab_list
View external linksLinks

Vocabulary list of GPT-4o (o200k_base) and GPT-4/GPT-3.5 (cl100k_base) tokenizers. Special tokens are excluded.
68May 14, 2024Updated last year

Alternatives and similar repositories for gpt4_vocab_list

Users that are interested in gpt4_vocab_list are comparing it to the libraries listed below

Sorting:

Are these results useful?