LLM Tokenizer with BPE algorithm
☆49May 7, 2024Updated last year
Alternatives and similar repositories for bpe-tokenizer
Users that are interested in bpe-tokenizer are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- ☆96Jul 20, 2025Updated 9 months ago
- A simple deep learning framework inspired by Dezero and PyTorch☆31Jan 27, 2025Updated last year
- MoE model with onnx runtime☆60May 5, 2024Updated 2 years ago
- ☆19Aug 9, 2024Updated last year
- Use pretrained BERT model to automatically generate grammar multiple choice questions (MCQ) from any news article or story.☆13Oct 2, 2019Updated 6 years ago
- 1-Click AI Models by DigitalOcean Gradient • AdDeploy popular AI models on DigitalOcean Gradient GPU virtual machines with just a single click. Zero configuration with optimized deployments.
- stay tuned.☆18Jul 7, 2025Updated 9 months ago
- (NBCE)Naive Bayes-based Context Extension on ChatGLM-6b☆15Jun 7, 2023Updated 2 years ago
- ☆60Mar 8, 2025Updated last year
- 高性能文本 Tokenizer 库☆31Feb 2, 2024Updated 2 years ago
- Retriever-0.1B☆96Jun 6, 2024Updated last year
- 基于CLIP实现以文精准搜图☆16Sep 20, 2023Updated 2 years ago
- 通义千问 SFT试验☆83Jan 6, 2024Updated 2 years ago
- 微调阿里开源的文字检测模型，利用合合识别返回的OCR结果作为初始训练数据，对模型进行优化训练，使其更加适应1万张图片的具体场景，提高文字识别的精度。☆10Dec 9, 2024Updated last year
- RAG向量召回示例☆153Feb 14, 2024Updated 2 years ago
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- Math-VR Benchmark & CodePlot-CoT: Mathematical Visual Reasoning by Thinking with Code-Driven Images☆59Nov 4, 2025Updated 6 months ago
- [ECCV'24] UNIT: Backdoor Mitigation via Automated Neural Distribution Tightening☆10Dec 18, 2025Updated 4 months ago
- 使用Qwen1.5-0.5B-Chat模型进行通用信息抽取任务的微调，旨在： 验证生成式方法相较于抽取式NER的效果； 为新手提供简易的模型微调流程，尽量减少代码量； 大模型训练的数据格式处理。☆14Sep 6, 2024Updated last year
- Taurix OS kernel. Taurix 系统内核，操作系统原理实(xjb)践(写)☆12Dec 20, 2020Updated 5 years ago
- An Efficent BPE Algorithm Faster then Hugging Face Tokenizer's Implementation☆13Sep 9, 2024Updated last year
- Debug DeepSpeed-Chat step by step in IDE (在IDE里一步一步调试DeepSpeed-Chat)☆10Apr 17, 2023Updated 3 years ago
- CUDA SGEMM optimization note☆15Oct 31, 2023Updated 2 years ago
- NewsApp包含客户端源码、服务端源码、数据库文件。 基于Miscrosoft人工智能项目ProjectOxford中的Recognition Emotion做的， 主要是基于用户的面部表情来推送不同类别的新闻。 Emotion API可以参考：https://www.p…☆10Mar 2, 2016Updated 10 years ago
- Code release for "Generating Code World Models with Large Language Models Guided by Monte Carlo Tree Search" published at NeurIPS '24.☆18Feb 21, 2025Updated last year
- Virtual machines for every use case on DigitalOcean • AdGet dependable uptime with 99.99% SLA, simple security tools, and predictable monthly pricing with DigitalOcean's virtual machines, called Droplets.
- ☆11May 24, 2023Updated 2 years ago
- Musculoskeletal Analysis extension for 3D Slicer. Currently has cortical, cancellous, and bone density analysis.☆12May 2, 2024Updated 2 years ago
- ☆21Jun 16, 2025Updated 10 months ago
- Automated detection of exudates from fundus images plays an important role in diabetic retinopathy (DR) screening and evaluation, for whi…☆11Dec 11, 2020Updated 5 years ago
- ☆45Mar 24, 2026Updated last month
- ☆13Mar 11, 2026Updated last month
- 基于DPO算法微调语言大模型，简单好上手。☆51Jul 3, 2024Updated last year
- 基于 BPE 实现的中文分词。优化：预处理，并行计算，多字词，多词表☆14May 14, 2022Updated 3 years ago
- ☆13Nov 10, 2024Updated last year
- Managed Database hosting by DigitalOcean • AdPostgreSQL, MySQL, MongoDB, Kafka, Valkey, and OpenSearch available. Automatically scale up storage and focus on building your apps.
- Tree-Invent: A novel molecular generative model constrained with topological tree☆14Jul 26, 2023Updated 2 years ago
- ICCV'23 Dual Learning with Dynamic Knowledge Distillation for Partially Relevant Video Retrieval☆19Aug 22, 2025Updated 8 months ago
- This project utilizes deep reinforcement learning techniques to train a robot, which combines a mobile platform and a Panda robotic arm, …☆10Jun 7, 2023Updated 2 years ago
- xgboost复现☆15Oct 6, 2024Updated last year
- ☆14Mar 28, 2025Updated last year
- Future version of the AnyBody Managed Model Repository with a full thoracic spine model.☆19Apr 15, 2026Updated 3 weeks ago
- ☆11Aug 10, 2023Updated 2 years ago