isLinXu / regex-tokenizerLinks
Converted the Jina Tokenizer regex pattern to python.
☆26Updated last year
Alternatives and similar repositories for regex-tokenizer
Users that are interested in regex-tokenizer are comparing it to the libraries listed below
Sorting:
- A Python Package to Access World-Class Generative Models☆129Updated last year
- 如需体验textin文档解析,请点击https://cc.co/16YSIy☆118Updated 2 months ago
- 顾名思义:手搓的RAG☆127Updated last year
- 利用免费的大模型api来结合你的私域数据来生成sft训练数据(妥妥白嫖)支持llamafactory等工具的训练数据格式synthetic data☆181Updated 9 months ago
- Analysis of Chinese and English layouts 中英文版面分析☆242Updated 3 weeks ago
- gpt_server是一个用于生产级部署LLMs、Embedding、Reranker、ASR和TTS的开源框架。☆206Updated last week
- unify-easy-llm(ULM)旨在打造一个简易的一键式大模型训练工具,支持Nvidia GPU、Ascend NPU等不同硬件以及常用的大模型。☆57Updated last year
- Agentica: Effortlessly Build Intelligent, Reflective, and Collaborative Multimodal AI Agents! 构建智能的多模态AI Agent。☆202Updated 2 weeks ago
- 大模型预训练中文语料清洗及质量评估 Large model pre-training corpus cleaning☆69Updated last year
- 中文原生检索增强生成测评基准☆122Updated last year
- 360LayoutAnaylsis, a series Document Analysis Models and Datasets deleveped by 360 AI Research Institute☆297Updated 11 months ago
- ☆67Updated 11 months ago
- 📝 针对文档类图像做内容提取,将文档类图像一比一输出到Word或者Txt中,便于进一步使用或处理。后续计划支持输入PDF/图像,输出对应json格式、Txt格式、Word格式和Markdown格式。☆204Updated 10 months ago
- Fast pdf translate是一款pdf翻译软件,基于MinerU实现pdf转markdown的功能,接着对markdown进行分割, 送给大模型翻译,最后组装翻译结果并由pypandoc生成结果pdf。☆29Updated 5 months ago
- Qwen DianJin: LLMs for the Financial Industry by Alibaba Cloud☆242Updated last week
- bisheng-unstructured library☆55Updated 3 months ago
- 基于Qwen2模型进行通用信息抽取【实体/关系/事件抽取】☆35Updated last year
- 探索 LLM 在法律行业的应用潜力☆91Updated 8 months ago
- 360zhinao☆291Updated 3 months ago
- Finetune Llama 3, Mistral & Gemma LLMs 2-5x faster with 80% less memory☆29Updated last year
- vLLM Documentation in Chinese Simplified / vLLM 中文文档☆94Updated 3 weeks ago
- 演示 vllm 对中文大语言模型的神奇效果☆31Updated last year
- 中文世界的NLP自动标注开源工具,简单样本,交给LabelFast。☆75Updated 7 months ago
- ☆106Updated last year
- 基于ReAct手搓一个Agent Demo☆147Updated 2 months ago
- Easy, fast, and cheap pretrain,finetune, serving for everyone☆314Updated last month
- SearchGPT: Building a quick conversation-based search engine with LLMs.☆47Updated 7 months ago
- 部署你自己的OpenAI api🤩, 基于flask, transformers (使用 Baichuan2-13B-Chat-4bits 模型, 可以运行在单张Tesla T4显卡) ,实现了OpenAI中Chat, Models和Completions接口,包含流式响…☆95Updated last year
- 利用LLM+敏感词库,来自动判别是否涉及敏感词。☆127Updated 2 years ago
- Based on RapidOCR, extract the PDF content☆182Updated 3 months ago