bojone / bytepiece
更纯粹、更高压缩率的Tokenizer
☆454Updated 7 months ago
Related projects ⓘ
Alternatives and complementary repositories for bytepiece
- ☆297Updated last year
- Train a 1B LLM with 1T tokens from scratch by personal☆347Updated last week
- Firefly中文LLaMA-2大模型，支持增量预训练Baichuan2、Llama2、Llama、Falcon、Qwen、Baichuan、InternLM、Bloom等大模型☆399Updated last year
- Baichuan2代码的逐行解析版本，适合小白☆208Updated last year
- 欢迎来到 "LLM-travel" 仓库！探索大语言模型（LLM）的奥秘 🚀。致力于深入理解、探讨以及实现与大模型相关的各种技术、原理和应用。☆269Updated 4 months ago
- 语言模型中文认知能力分析☆235Updated last year
- 使用sentencepiece中BPE训练中文词表，并在transformers中进行使用。☆111Updated last year
- 中文图书语料MD5链接☆212Updated 9 months ago
- 大模型多维度中文对齐评测基准 (ACL 2024)☆334Updated 3 months ago
- BiLLa: A Bilingual LLaMA with Enhanced Reasoning Ability☆421Updated last year
- 开源SFT数据集整理,随时补充☆447Updated last year
- The official repo of Aquila2 series proposed by BAAI, including pretrained & chat large language models.☆438Updated last month
- OpenLLMWiki: Docs of OpenLLMAI. Survey, reproduction and domain/task adaptation of open source chatgpt alternatives/implementations. PiXi…☆248Updated 8 months ago
- ☆158Updated last year
- FlagEval is an evaluation toolkit for AI large foundation models.☆302Updated 4 months ago
- alpaca中文指令微调数据集☆391Updated last year
- text embedding☆140Updated last year
- 大语言模型指令调优工具（支持 FlashAttention）☆166Updated 10 months ago
- ☆213Updated 6 months ago
- pCLUE: 1000000+多任务提示学习数据集☆469Updated 2 years ago
- Chinese large language model base generated through incremental pre-training on Chinese datasets☆234Updated last year
- ☆273Updated 6 months ago
- 中文 Instruction tuning datasets☆118Updated 7 months ago
- A Chinese Open-Domain Dialogue System☆314Updated last year
- Implementation of Chinese ChatGPT☆287Updated last year
- This is a repository used by individuals to experiment and reproduce the pre-training process of LLM.☆356Updated 7 months ago
- ChatGLM-6B 指令学习|指令数据|Instruct☆652Updated last year
- Collaborative Training of Large Language Models in an Efficient Way☆411Updated 2 months ago
- 中文书籍收录整理, Collection of Chinese Books☆173Updated 10 months ago
- Naive Bayes-based Context Extension☆313Updated last year