SharpToken is a C# library for tokenizing natural language text. It's based on the tiktoken Python library and designed to be fast and accurate.
☆263Mar 25, 2026Updated 6 months ago
Alternatives and similar repositories for SharpToken
Users that are interested in SharpToken are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- High-performance .NET BPE tokenizer — up to 618 MiB/s, competitive with Rust. Zero-allocation counting, multilingual cache, o200k/cl100k/…☆92Sep 25, 2026Updated 2 weeks ago
- A collection of tools for working with OpenAI☆102May 4, 2023Updated 3 years ago
- Typescript and .NET implementation of BPE tokenizer for OpenAI LLMs.