DezhiKong00 / Sentencepiece-chinese-bbpe

Related projects ⓘ

Alternatives and complementary repositories for Sentencepiece-chinese-bbpe

• mobvoi / seq-monkey-data
  ☆92Updated 6 months ago
• QwenLM / vllm-gptq
  A high-throughput and memory-efficient inference and serving engine for LLMs
  ☆124Updated 11 months ago
• the-seeds / imitater
  Imitate OpenAI with Local Models
  ☆85Updated 2 months ago
• yanqiangmiffy / how-to-train-tokenizer
  怎么训练一个LLM分词器
  ☆130Updated last year
• lilongxian / BaiYang-chatGLM2-6B
  （1）弹性区间标准化的旋转位置词嵌入编码器+peft LORA量化训练，提高万级tokens性能支持。（2）证据理论解释学习，提升模型的复杂逻辑推理能力（3）兼容alpaca数据格式。
  ☆45Updated last year
• CASIA-LM / ChineseWebText
  ☆158Updated last year
• wp931120 / baichuan_sft_lora
  baichuan LLM surpervised finetune by lora
  ☆60Updated last year
• YouTaoBaBa / Chinese-Dialogue-Dataset
  用于汇总目前的开源中文对话数据集
  ☆116Updated last year
• mark1879 / Baichuan-13B-Finetuning
  Baichuan-13B 指令微调
  ☆89Updated last year
• NEU-DataMining / PICA
  多轮共情对话模型PICA
  ☆86Updated last year
• morecry / CharacterEval
  ☆181Updated this week
• jingzhunxue / flow_mirror
  flow mirror models from JZX AI Labs
  ☆40Updated last month
• ProjectD-AI / llama_inference
  llama inference for tencentpretrain
  ☆96Updated last year
• ssbuild / deep_training
  deep learning
  ☆149Updated 5 months ago
• ArtificialZeng / Baichuan2-Explained
  Baichuan2代码的逐行解析版本，适合小白
  ☆208Updated last year
• taishan1994 / sentencepiece_chinese_bpe
  使用sentencepiece中BPE训练中文词表，并在transformers中进行使用。
  ☆110Updated last year
• SupritYoung / RLHF-Label-Tool
  用于大模型 RLHF 进行人工数据标注排序的工具。A tool for manual response data annotation sorting in RLHF stage.
  ☆243Updated last year
• ssbuild / qwen_finetuning
  qwen-7b and qwen-14b finetuning
  ☆84Updated 7 months ago
• Suffoquer-fang / LuXun-GPT
  LLM with LuXun (鲁迅) style
  ☆78Updated last year
• seanzhang-zhichen / Qwen-WisdomVast
  Qwen-WisdomVast is a large model trained on 1 million high-quality Chinese multi-turn SFT data, 200,000 English multi-turn SFT data, and …
  ☆18Updated 7 months ago
• beichao1314 / Open-Llama
  The complete training code of the open-source high-performance Llama model, including the full process from pre-training to RLHF.
  ☆61Updated last year
• Longyichen / Alpaca-family-library
  Summarize all open source Large Languages Models and low-cost replication methods for Chatgpt.
  ☆135Updated last year
• stanleylsx / llms_tool
  一个基于HuggingFace开发的大语言模型训练、测试工具。支持各模型的webui、终端预测，低参数量及全参数模型训练(预训练、SFT、RM、PPO、DPO)和融合、量化。
  ☆203Updated 11 months ago
• wangwenju269 / work_space
  NLP 项目记录档案
  ☆43Updated 3 weeks ago
• shibing624 / lmft
  ChatGLM-6B fine-tuning.
  ☆135Updated last year
• SmartLi8 / stella
  text embedding
  ☆139Updated last year
• OpenSenseNova / piccolo-embedding
  code for piccolo embedding model from SenseTime
  ☆111Updated 6 months ago
• yangjianxin1 / LongQLoRA
  LongQLoRA: Extent Context Length of LLMs Efficiently
  ☆159Updated last year
• jiahe7ay / MiniCharacterLLM
  这是一个一键让小参数大模型进行角色扮演的项目，从数据构成和训练都包含在这项目中
  ☆16Updated 7 months ago
• aplmikex / deduplication_mnbvc
  文本去重
  ☆67Updated 6 months ago