howl-anderson / MicroTokenizerLinks

一个轻量且功能全面的中文分词器,帮助学生了解分词器的工作原理。MicroTokenizer: A lightweight Chinese tokenizer designed for educational and research purposes. Provides a practical, hands-on approach to understanding NLP concepts, featuring multiple tokenization algorithms and customizable models. Ideal for students, researchers, and NLP enthusiasts..
157Updated 10 months ago

Alternatives and similar repositories for MicroTokenizer

Users that are interested in MicroTokenizer are comparing it to the libraries listed below

Sorting: