isLinXu / regex-tokenizerLinks
Converted the Jina Tokenizer regex pattern to python.
☆26Updated 11 months ago
Alternatives and similar repositories for regex-tokenizer
Users that are interested in regex-tokenizer are comparing it to the libraries listed below
Sorting:
- A Python Package to Access World-Class Generative Models☆128Updated last year
- 如需体验textin文档解析,请点击https://cc.co/16YSIy☆114Updated last month
- unify-easy-llm(ULM)旨在打造 一个简易的一键式大模型训练工具,支持Nvidia GPU、Ascend NPU等不同硬件以及常用的大模型。☆56Updated last year
- gpt_server是一个用于生产级部署LLMs、Embedding、Reranker、ASR和TTS的开源框架。☆204Updated 2 weeks ago
- 中文原生检索增强生成测评基准☆120Updated last year
- Analysis of Chinese and English layouts 中英文版面分析☆235Updated this week
- 顾名思义:手搓的RAG☆125Updated last year
- 360LayoutAnaylsis, a series Document Analysis Models and Datasets deleveped by 360 AI Research Institute☆293Updated 11 months ago
- 视觉信息抽取任务中,使用OCR识别结果规范多模态大模型的回答☆39Updated 7 months ago
- 利用免费的大模型api来结合你的私域数据来生成sft训练数据(妥妥白嫖)支持llamafactory等工具的训练数据格式synthetic data☆176Updated 8 months ago
- 利用LLM+敏感词库,来自动判别是否涉及敏感词。☆126Updated 2 years ago
- Agentica: Effortlessly Build Intelligent, Reflective, and Collaborative Multimodal AI Agents! 构建智能的多模态AI Agent。☆198Updated 2 weeks ago
- ☆66Updated 10 months ago
- ☆106Updated last year
- Based on RapidOCR, extract the PDF content☆179Updated 3 months ago
- 本项目借助飞桨平台,构建起一套创新的多模型协同系统,实现 PDF 文件到 Markdown 文件的高效、精准转换。☆22Updated 4 months ago
- 大模型预训练中文语料清洗及质量评估 Large model pre-training corpus cleaning☆67Updated last year
- vLLM Documentation in Chinese Simplified / vLLM 中文文档☆90Updated 2 months ago
- SearchGPT: Building a quick conversation-based search engine with LLMs.☆47Updated 7 months ago
- 探索 LLM 在法律行业的应用潜力☆91Updated 8 months ago
- ☆231Updated last year
- 纯c++的全平台llm加速库,支持python调用,支持baichuan, glm, llama, moss基座,手机端流畅运行chatglm-6B级模型单卡可达10000+token / s,☆45Updated last year
- 360zhinao☆290Updated 2 months ago
- Fast pdf translate是一款pdf翻译软件,基于MinerU实现pdf转markdown的功能,接着对markdown进行分割, 送给大模型翻译,最后组装翻译结果并由pypandoc生成结果pdf。☆28Updated 4 months ago
- Alpaca Chinese Dataset -- 中文指令微调数据集☆213Updated 10 months ago
- Easy, fast, and cheap pretrain,finetune, serving for everyone☆313Updated 3 weeks ago
- ☆349Updated last year
- bisheng-unstructured library☆55Updated 2 months ago
- A high-throughput and memory-efficient inference and serving engine for LLMs☆137Updated 8 months ago
- ☆127Updated 4 months ago