LLM Tokenizer with BPE algorithm
☆47May 7, 2024Updated last year
Alternatives and similar repositories for bpe-tokenizer
Users that are interested in bpe-tokenizer are comparing it to the libraries listed below
Sorting:
- simple decoder-only GTP model in pytorch☆43May 19, 2024Updated last year
- A simple, easy-to-hack GraphRAG implementation☆15Sep 21, 2024Updated last year
- A simple deep learning framework inspired by Dezero and PyTorch☆31Jan 27, 2025Updated last year
- 通义千问的DPO训练☆63Sep 21, 2024Updated last year
- ☆93Jul 20, 2025Updated 7 months ago
- 使用Sentencepiece对中文语料进行分词☆13Nov 30, 2023Updated 2 years ago
- 微调阿里开源的文字检测模型，利用合合识别返回的OCR结果作为初始训练数据，对模型进行优化训练，使其更加适应1万张图片的具体场景，提高文字识别的精度。☆10Dec 9, 2024Updated last year
- 使用Qwen1.5-0.5B-Chat模型进行通用信息抽取任务的微调，旨在： 验证生成式方法相较于抽取式NER的效果； 为新手提供简易的模型微调流程，尽量减少代码量； 大模型训练的数据格式处理。☆15Sep 6, 2024Updated last year
- ☆11Feb 25, 2026Updated last week
- ☆120Jun 30, 2024Updated last year
- ☆19Aug 9, 2024Updated last year
- bilibili视频讲解所使用的课件代码记录☆26Sep 23, 2025Updated 5 months ago
- ☆59Mar 8, 2025Updated 11 months ago
- MoE model with onnx runtime☆59May 5, 2024Updated last year
- Musculoskeletal Analysis extension for 3D Slicer. Currently has cortical, cancellous, and bone density analysis.☆12May 2, 2024Updated last year
- Future version of the AnyBody Managed Model Repository with a full thoracic spine model.☆18Updated this week
- 基于改进YOLOv8与DeepSeek微调的智能交通监控与问答系统☆19Apr 16, 2025Updated 10 months ago
- Qwen1.5-SFT(阿里, Ali), Qwen_Qwen1.5-2B-Chat/Qwen_Qwen1.5-7B-Chat微调(transformers)/LORA(peft)/推理☆74May 17, 2024Updated last year
- Difyで作る生成AIアプリ完全入門☆17May 25, 2025Updated 9 months ago
- ☆23Updated this week
- A simple WeChat Official Account layout tool based on Dify☆17Jun 27, 2025Updated 8 months ago
- ☆76Nov 13, 2023Updated 2 years ago
- 高性能文本 Tokenizer 库☆32Feb 2, 2024Updated 2 years ago
- ☆13Nov 10, 2024Updated last year
- Write the database metadata into the dify knowledge☆12Dec 30, 2025Updated 2 months ago
- ☆17Feb 6, 2025Updated last year
- A full-stack AI-powered business intelligence tool for non-experts, featuring serverless backend processing and a secure Streamlit fronte…☆28Feb 13, 2026Updated 3 weeks ago
- 参考《上海交通大学生存手册》开源☆16Sep 25, 2024Updated last year
- ☆28Dec 4, 2025Updated 3 months ago
- Workflow automation, but you just describe what you want and it happens.☆27Nov 22, 2025Updated 3 months ago
- ☆11Apr 17, 2025Updated 10 months ago
- ☆11Aug 29, 2025Updated 6 months ago
- 基于Qwen2+SFT+DPO的医疗问答系统，项目中使用了自定义的 SFTTrainer/DPOTrainer/TRPOTrainer用于训练，其次，项目还调用各种知识库工具（neo4j, milvus, LDA, 等）进行自动化训练数据生成。另外，使用 vllm 用于推理…☆61Jan 4, 2026Updated 2 months ago
- Tree-Invent: A novel molecular generative model constrained with topological tree☆13Jul 26, 2023Updated 2 years ago
- 通义千问 SFT试验☆81Jan 6, 2024Updated 2 years ago
- A high-throughput and memory-efficient inference and serving engine for LLMs☆37Jan 26, 2025Updated last year
- 基于DPO算法微调语言大模型，简单好上手。☆50Jul 3, 2024Updated last year
- Debug DeepSpeed-Chat step by step in IDE (在IDE里一步一步调试DeepSpeed-Chat)☆10Apr 17, 2023Updated 2 years ago
- 🤖AI Agents for Financial Trading💰: LLM-Driven Stock Prediction & Investment Recommendation System☆13Apr 14, 2025Updated 10 months ago