Alternatives and similar repositories for bertTokenizer

Users that are interested in bertTokenizer are comparing it to the libraries listed below

• ankiteciitkgp / bertTokenizer
  A java implementation of Bert Tokenizer.
  ☆29Updated 4 years ago
• mobvoi / seq-monkey-data
  ☆175Updated last year
• xinsblog / chatglm-tiny
  从头开始训练一个chatglm小模型
  ☆49Updated 2 years ago
• yeyupiaoling / PunctuationModel
  中文标点符号模型，可以给文本添加标点符号。
  ☆147Updated last year
• jiangnanboy / jcorrector
  jcorrector 中文文本纠错工具， Text Error Correction Tool，Spelling Check
  ☆81Updated 11 months ago
• open-chinese / alpaca-chinese-dataset
  Alpaca Chinese Dataset -- 中文指令微调数据集
  ☆216Updated last year
• taishan1994 / sentencepiece_chinese_bpe
  使用sentencepiece中BPE训练中文词表，并在transformers中进行使用。
  ☆120Updated 2 years ago
• chenyangMl / JointBERT-zh
  Pytorch implementation of JointBERT: "BERT for Joint Intent Classification and Slot Filling"
  ☆46Updated 2 years ago
• ssbuild / qwen_finetuning
  qwen models finetuning
  ☆106Updated 11 months ago
• XiaoMi / subllm
  This repository is the official implementation of the ECAI 2024 conference paper SUBLLM: A Novel Efficient Architecture with Token Sequen…
  ☆68Updated last year
• wangyingdong / m3e-base
  ☆68Updated 2 years ago
• YouTaoBaBa / Chinese-Dialogue-Dataset
  用于汇总目前的开源中文对话数据集
  ☆200Updated 3 years ago
• QwenLM / vllm-gptq
  A high-throughput and memory-efficient inference and serving engine for LLMs
  ☆140Updated last year
• chenyangMl / llama2.c-zh
  支持中文场景的的小语言模型 llama2.c-zh
  ☆150Updated last year
• Executedone / ChatGLM4Tools
  chatglm-6B for tools application using langchain
  ☆76Updated 2 years ago
• ProjectD-AI / llama_inference
  llama inference for tencentpretrain
  ☆99Updated 2 years ago
• Ninot1Quyi / Qwen2.5-Omni-multimodal-chat
  基于通义千问 Qwen2.5-Omni 的实时语音对话系统，使用在线API服务，支持实时语音交互、动态语音活动检测和流式音频处理。A real-time voice conversation system based on Qwen2.5-Omni Online-API, …
  ☆83Updated 9 months ago
• lovemefan / telespeech-asr-python
  ☆69Updated last year
• jiangnanboy / llm_corpus_quality
  大模型预训练中文语料清洗及质量评估 Large model pre-training corpus cleaning
  ☆75Updated last year
• ChuangLee / ChatGLM-6B-multiGPU
  多显卡部署版 | ChatGLM-6B：开源双语对话语言模型 | An Open Bilingual Dialogue Language Model
  ☆62Updated 2 years ago
• shuliu586 / AI_Chinese_DataSet_KnowledgeDAO
  供AI训练的中文数据集（持续更新。。。）与AI公司图谱，目前的数据集餐饮行业8000问，百度知道，Alpaca中文数据集，计算机领域数据集，Vicuna数据集，RedPajama数据集，Wikipedia中文词条数据集，网站论坛问答数据集
  ☆64Updated 2 years ago
• wangshuai09 / vllm
  A high-throughput and memory-efficient inference and serving engine for LLMs
  ☆39Updated 5 months ago
• xinchen-ai / Westlake-Omni
  ☆204Updated last year
• shjwudp / shu
  中文书籍收录整理, Collection of Chinese Books
  ☆206Updated 2 years ago
• znMemories / ChatGLM-Dataset-Maker
  适用于ChatGLM微调的数据集生成器, 支持多轮对话
  ☆15Updated 2 years ago
• yongzhuo / chatglm-maths
  chatglm-6b微调/LORA/PPO/推理, 样本为自动生成的整数/小数加减乘除运算, 可gpu/cpu
  ☆165Updated 2 years ago
• chenyangMl / keyword-spot
  端到端语音唤醒工具箱，从模型训练到模型推理。
  ☆154Updated 6 months ago
• git-cloner / Llama2-chinese
  Llama2 chinese finetuning
  ☆38Updated 2 years ago
• lovemefan / CT-Transformer-punctuation
  A enterprise-grade Chinese-English code switch punctuator from funasr.
  ☆30Updated last year
• yongzhuo / qwen2-sft
  Qwen1.5-SFT(阿里, Ali), Qwen_Qwen1.5-2B-Chat/Qwen_Qwen1.5-7B-Chat微调(transformers)/LORA(peft)/推理
  ☆71Updated last year