isLinXu / regex-tokenizer
Converted the Jina Tokenizer regex pattern to python.
☆26Updated 5 months ago
Alternatives and similar repositories for regex-tokenizer:
Users that are interested in regex-tokenizer are comparing it to the libraries listed below
- ☆62Updated 4 months ago
- Analysis of Chinese and English layouts 中英文版面分析☆160Updated last month
- 中文原生检索增强生成测评基准☆107Updated 9 months ago
- 如需体验textin文档解析,请点击https://cc.co/16YSIy☆72Updated 2 months ago
- 顾名思义:手搓的RAG☆116Updated 11 months ago
- 360LayoutAnaylsis, a series Document Analysis Models and Datasets deleveped by 360 AI Research Institute☆261Updated 4 months ago
- bisheng-unstructured library☆41Updated 2 months ago
- Python3 package for Chinese/English OCR, with paddleocr-v4 onnx model(~14MB). 基于ppocr-v4-onnx模型推理,可实现 CPU 上毫秒级的 OCR 精准预测,通用场景中英文OCR达到开源SO…☆55Updated last week
- gpt_server是一个用于生产级部署LLMs或Embedding的开源框架。☆148Updated 2 weeks ago
- dify's rag patch module☆99Updated last week
- 利用免费的大模型api来结合你的私域数据来生成sft训练数据(妥妥白嫖)支持llamafactory等工具的训练数据格式synthetic data☆119Updated 2 months ago
- 中文世界的NLP自动标注开源工具,简单样本,交给LabelFast。☆62Updated 3 weeks ago
- A Python Package to Access World-Class Generative Models☆127Updated 7 months ago
- ☆105Updated last year
- 纯c++的全平台llm加速库,支持python调用,支持baichuan, glm, llama, moss基座,手机端流畅运行chatglm-6B级模型单卡可达10000+token / s,☆45Updated last year
- ☆107Updated 5 months ago
- 演示 vllm 对中文大语言模型的神奇效果☆31Updated last year
- Finetune Llama 3, Mistral & Gemma LLMs 2-5x faster with 80% less memory☆25Updated 8 months ago
- 📝 针对文档类图像做内容提取,将文档类图像一比一输出到Word或者Txt中,便于进一步使用或处理。后续计划支持输入PDF/图像,输出对应json格式、Txt格式、Word格式和Markdown格式。☆178Updated 2 months ago
- A demo built on Megrez-3B-Instruct, integrating a web search tool to enhance the model's question-and-answer capabilities.☆35Updated last month
- 通用版面分析 | 中文文档解析 |Document Layout Analysis | layout paser☆45Updated 7 months ago
- 基于Qwen2模型进行通用信息抽取【实体/关系/事件抽取】☆29Updated 6 months ago
- ☆114Updated 8 months ago
- 百度QA100万数据集☆47Updated last year
- 基于ReAct手搓一个Agent Demo☆112Updated 9 months ago
- 大模型预训练中文语料清洗及质量评 估 Large model pre-training corpus cleaning☆51Updated 6 months ago
- 本项目致力于为大模型领域的初学者提供全面的知识体系,包括基础和高阶内容,以便开发者能迅速掌握大模型技术栈并全面了解相关知识。☆44Updated 3 weeks ago
- unify-easy-llm(ULM)旨在打造一个简易的一键式大模型训练工具,支持Nvidia GPU、Ascend NPU等不同硬件以及常用的大模型。☆54Updated 6 months ago
- SMP 2023 ChatGLM金融大模型挑战赛 60 分baseline思路介绍☆184Updated last year
- The Level-Navi Agent, a framework that requires no training and utilizes large language models for deep query understanding and precise s…☆11Updated last month