howl-anderson / MicroTokenizer
一个轻量且功能全面的中文分词器,帮助学生了解分词器的工作原理。MicroTokenizer: A lightweight Chinese tokenizer designed for educational and research purposes. Provides a practical, hands-on approach to understanding NLP concepts, featuring multiple tokenization algorithms and customizable models. Ideal for students, researchers, and NLP enthusiasts..
☆146Updated 3 months ago
Alternatives and similar repositories for MicroTokenizer:
Users that are interested in MicroTokenizer are comparing it to the libraries listed below
- Simple Solution for Multi-Criteria Chinese Word Segmentation☆300Updated 4 years ago
- 各大中文分词性能评测☆155Updated 6 years ago
- 中文近义词表 Chinese Synonyms☆250Updated 7 years ago
- Train Wikidata with word2vec for word embedding tasks☆122Updated 6 years ago
- This is a corpus of Chinese abbreviation, including negative full forms.☆190Updated 3 years ago
- Self complemented Pinyin2Chinese demo use algorithms including Trie and HMM model , 基于隐马尔科夫模型与Trie树的拼音切分与拼音转中文的简单demo实现。☆86Updated 6 years ago
- THU Chinese Keyphrase Extraction Toolkit☆124Updated 6 years ago
- 基于最小熵原理的NLP工具包☆138Updated 3 years ago
- 李傲龍的博客☆81Updated 6 months ago
- ChineseTextualInference project including chinese corpus build and inferecence model, 中文文本推断项目,包括88万文本蕴含中文文本蕴含数据集的翻译与构建,基于深度学习的文本蕴含判定模型构建…☆168Updated 6 years ago
- A curated list of resources of chinese corpora for NLP(Natural Language Processing)☆74Updated 5 years ago
- An collection of Chinese nlp corpus including basic Chinese syntatic wordset, semantic wordset, historic corpus and evaluate corpus. 中文自然…☆440Updated 6 years ago
- 教育行业新闻 自动文摘 语料库 自动摘要☆196Updated 6 years ago
- chinese anti semantic word search interface based on dict crawled from online resources, ChineseAntiword,针对中文词语的反义词查询接口☆59Updated 6 years ago
- SemEval-2016 Task 9: Chinese Semantic Dependency Parsing☆135Updated 6 years ago
- 新词发现算法(NewWordDetection)☆92Updated 3 years ago
- Corpus creator for Chinese Wikipedia☆41Updated 3 years ago
- AbstractKnowledgeGraph, a systematic knowledge graph that concentrate on abstract thing including abstract entity and action. 抽象知识图谱,目前规模…☆239Updated 5 years ago
- 夸夸语料,来自豆瓣互相表扬组数据☆75Updated 5 years ago
- 中文相关词典和语料库。☆169Updated 10 years ago
- 中文生成式预训练模型☆98Updated 4 years ago
- 依存关系分析,NLP,自然语言处理☆85Updated 3 years ago
- This directory contains the training, test, and gold-standard data used in the 2nd International Chinese Word Segmentation Bakeoff. Also …☆66Updated 6 years ago
- chinese and english corpus process script, python, c++, java☆194Updated 6 years ago
- 速度更快、效果更好的中文新词发现☆510Updated 11 months ago
- NLU is hard!!!☆270Updated 5 years ago
- NLP NER datasets video/music/book bio☆84Updated 4 years ago
- Code for chinese error detection module, using n-gram and bi-lstm☆131Updated 5 years ago
- 图书名语料库。含部分电影、游戏名称。☆68Updated 10 months ago
- Some useful Chinese corpus datasets 中文语料小数据☆529Updated 4 years ago