howl-anderson / MicroTokenizerLinks
一个轻量且功能全面的中文分词器，帮助学生了解分词器的工作原理。MicroTokenizer: A lightweight Chinese tokenizer designed for educational and research purposes. Provides a practical, hands-on approach to understanding NLP concepts, featuring multiple tokenization algorithms and customizable models. Ideal for students, researchers, and NLP enthusiasts..
☆153Updated 7 months ago
Alternatives and similar repositories for MicroTokenizer
Users that are interested in MicroTokenizer are comparing it to the libraries listed below
Sorting:
- Self complemented Pinyin2Chinese demo use algorithms including Trie and HMM model , 基于隐马尔科夫模型与Trie树的拼音切分与拼音转中文的简单demo实现。☆86Updated 7 years ago
- Code for chinese error detection module, using n-gram and bi-lstm☆135Updated 6 years ago
- 各大中文分词性能评测☆157Updated 6 years ago
- 基于最小熵原理的NLP工具包☆137Updated 3 years ago
- Train Wikidata with word2vec for word embedding tasks☆123Updated 6 years ago
- This is a corpus of Chinese abbreviation, including negative full forms.☆196Updated 3 years ago
- NLU is hard!!!☆273Updated 6 years ago
- 中文相关词典和语料库。☆174Updated 10 years ago
- 最全中华古诗词数据库, 唐宋两朝近一万四千古诗人, 接近5.5万首唐诗加26万宋诗. 两宋时期1564位词人，21050首词。☆59Updated 7 years ago
- 李傲龍的博客☆81Updated 10 months ago
- 中文近义词表 Chinese Synonyms☆258Updated 7 years ago
- 教育行业新闻 自动文摘 语料库 自动摘要☆200Updated 7 years ago
- Time-NLP的python3版本 中文时间表达词转换☆517Updated 2 years ago
- Corpus creator for Chinese Wikipedia☆41Updated 3 years ago
- 汉字字符特征提取器 (featurizer)，提取汉字的特征（发音特征、字形特征）用做深度学习的特征 ｜ A Chinese character feature extractor, which extracts the features of Chinese charac…☆295Updated 4 years ago
- 书籍《现代自然语言生成》介绍☆218Updated 4 years ago
- Simple Solution for Multi-Criteria Chinese Word Segmentation☆302Updated 4 years ago
- 基于哈工大同义词词林扩展版的单词相似度计算方法☆366Updated 2 years ago
- 简易的中文纠错和消歧☆289Updated 9 years ago
- 天池nl2sql冠军方案☆301Updated 5 years ago
- 依存关系分析，NLP，自然语言处理☆85Updated 3 years ago
- 速度更快、效果更好的中文新词发现☆511Updated last year
- AbstractKnowledgeGraph, a systematic knowledge graph that concentrate on abstract thing including abstract entity and action. 抽象知识图谱，目前规模…☆246Updated 5 years ago
- Some useful Chinese corpus datasets 中文语料小数据☆534Updated 5 years ago
- 中文文本自动纠错☆85Updated 6 years ago
- 最好的汉字数字(中文数字)-阿拉伯数字转换工具。包含"点二八"，"负百分之四十"等众多汉语表达方法。NLP，机器人工程必备！ The Best Tool of Chinese Number to Digits☆367Updated 2 years ago
- 一个中文的已标注词性的语料库☆203Updated 10 years ago
- ChineseTextualInference project including chinese corpus build and inferecence model, 中文文本推断项目,包括88万文本蕴含中文文本蕴含数据集的翻译与构建,基于深度学习的文本蕴含判定模型构建…☆172Updated 6 years ago
- NLP NER datasets video/music/book bio☆88Updated 4 years ago
- 中文生成式预训练模型☆98Updated 4 years ago