owenliang / bpe-tokenizer
LLM Tokenizer with BPE algorithm
☆27Updated 8 months ago
Alternatives and similar repositories for bpe-tokenizer:
Users that are interested in bpe-tokenizer are comparing it to the libraries listed below
- simple decoder-only GTP model in pytorch☆35Updated 7 months ago
- DeepSpeed Tutorial☆92Updated 5 months ago
- Inference code for LLaMA models☆114Updated last year
- ☆38Updated 5 months ago
- ☆97Updated 6 months ago
- Qwen1.5-SFT(阿里, Ali), Qwen_Qwen1.5-2B-Chat/Qwen_Qwen1.5-7B-Chat微调(transformers)/LORA(peft)/推理☆50Updated 8 months ago
- 一些大语言模型和多模态模型的应用,主要包括Rag，小模型，Agent，跨模态搜索，OCR等等☆144Updated 2 months ago
- LLM101n: Let's build a Storyteller 中文版☆121Updated 5 months ago
- Zero-human, cold-start construction of long-chain agents in professional domains☆20Updated this week
- ☆17Updated 5 months ago
- WWW2025 Multimodal Intent Recognition for Dialogue Systems Challenge☆114Updated 2 months ago
- MoE model with onnx runtime☆27Updated 8 months ago
- 欢迎来到 "LLM-travel" 仓库！探索大语言模型（LLM）的奥秘 🚀。致力于深入理解、探讨以 及实现与大模型相关的各种技术、原理和应用。☆283Updated 5 months ago
- 大语言模型应用：RAG、NL2SQL、聊天机器人、预训练、MOE混合专家模型、微调训练、强化学习、天池数据竞赛☆52Updated last month
- ☆61Updated last week
- DeepSpeed教程 & 示例注释 & 学习笔记 （大模型高效训练）☆146Updated last year
- ☆62Updated 2 months ago
- 通义千问的DPO训练☆30Updated 3 months ago
- 快速入门RAG与私有化部署☆144Updated 9 months ago
- 天池算法比赛《BetterMixture - 大模型数据混合挑战赛》的第一名top1解决方案☆24Updated 6 months ago
- Build a simple basic multimodal large model from scratch. 从零搭建一个简单的基础多模态大模型🤖☆27Updated 6 months ago
- A simple, easy-to-hack GraphRAG implementation☆9Updated 3 months ago
- 专注于对话系统领域的技术分享，重点写《Dify应用操作和源码剖析》专栏。☆71Updated 6 months ago
- ChatGLM-6B添加了RLHF的实现，以及部分核心代码的逐行讲解 ,实例部分是做了个新闻短标题的生成，以及指定context推荐的RLHF的实现☆81Updated last year
- Music large model based on InternLM2-chat.☆21Updated 3 weeks ago
- 使用单个24G显卡，从0开始训练LLM☆50Updated 2 months ago
- TinyRAG☆252Updated 2 months ago
- 从0开始，将chatgpt的技术路线跑一遍。☆184Updated 4 months ago
- 一个很小很小的RAG系统☆111Updated last month
- ☆22Updated last year