zejunwang1 / easytokenizerLinks
高性能文本 Tokenizer 库
☆31Updated last year
Alternatives and similar repositories for easytokenizer
Users that are interested in easytokenizer are comparing it to the libraries listed below
Sorting:
- BERT Tokenizer in C++☆78Updated 4 years ago
- 一个基于预训练的句向量生成工具☆138Updated 2 years ago
- text embedding☆147Updated 2 years ago
- code and data for "CSCD-NS: a Chinese Spelling Check Dataset for Native Speakers"☆76Updated last year
- sentence-transformers to onnx 让sbert模型推理效率更快☆167Updated 3 years ago
- 纯c++的全平台llm加速库，支持python调用，支持baichuan, glm, llama, moss基座，手机端流畅运行chatglm-6B级模型单卡可达10000+token / s，☆45Updated 2 years ago
- A PyTorch-based model pruning toolkit for pre-trained language models☆390Updated 2 years ago
- 基于模板的文本纠错；Automatically Mining Error Templates for Grammatical Error Correction☆44Updated 3 years ago
- llama inference for tencentpretrain☆99Updated 2 years ago
- 一个非常高效的字符串匹配工具，支持正向/反向最大匹配分词和多模式字符串精确匹配☆16Updated 2 years ago
- 使用sentencepiece中BPE训练中文词表，并在transformers中进行使用。☆120Updated 2 years ago
- ☆90Updated 2 years ago
- Code & Data for our Paper "NaSGEC: Multi-Domain Chinese Grammatical Error Correction for Native Speaker Texts" (ACL 2023 Findings)☆95Updated 9 months ago
- ChatGLM2-6B微调, SFT/LoRA, instruction finetune☆110Updated 2 years ago
- ChatGLM-6B fine-tuning.☆137Updated 2 years ago
- A more efficient GLM implementation!☆54Updated 2 years ago
- 中文 Instruction tuning datasets☆140Updated last year
- LORA微调BLOOMZ， 参考BELLE☆25Updated 2 years ago
- 百川Dynamic NTK-ALiBi的代码实现：无需微调即可推理更长文本☆49Updated 2 years ago
- Another ChatGLM2 implementation for GPTQ quantization☆53Updated 2 years ago
- 大规模中文语料☆44Updated 6 years ago
- 文本智能校对大赛(Chinese Text Correction)的baseline☆68Updated 3 years ago
- NLU & NLG (zero-shot) depend on mengzi-t5-base-mt pretrained model☆76Updated 3 years ago
- CCL 2022 汉语学习者文本纠错评测☆142Updated 2 years ago
- Source code for the paper "PLOME: Pre-training with Misspelled Knowledge for Chinese Spelling Correction" in ACL2021☆239Updated 3 years ago
- 🌈 NERpy: Implementation of Named Entity Recognition using Python. 命名实体识别工具，支持BertSoftmax、BertSpan等模型，开箱即用。☆116Updated last year
- A high-throughput and memory-efficient inference and serving engine for LLMs☆138Updated 11 months ago
- The Corpus & Code for EMNLP 2022 paper "FCGEC: Fine-Grained Corpus for Chinese Grammatical Error Correction" | FCGEC中文语法纠错语料及STG模型☆120Updated 11 months ago
- TensorRT☆11Updated 5 years ago
- ☆84Updated 2 years ago