tiktoken-go / tokenizerView on GitHub
Pure Go implementation of OpenAI's tiktoken tokenizer
444Jul 17, 2026Updated this week

Alternatives and similar repositories for tokenizer

Users that are interested in tokenizer are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.

Sorting:

Are these results useful?