Alternatives and similar repositories for regex-tokenizer

Users that are interested in regex-tokenizer are comparing it to the libraries listed below

• wangyuxinwhy / generate
  A Python Package to Access World-Class Generative Models
  ☆131Updated last year
• intsig-textin / markdown_tester
  如需体验textin文档解析，请点击https://cc.co/16YSIy
  ☆124Updated 6 months ago
• SmartFlowAI / Hand-on-RAG
  顾名思义：手搓的RAG
  ☆131Updated last year
• shell-nlp / gpt_server
  gpt_server是一个用于生产级部署LLMs、Embedding、Reranker、ASR、TTS、文生图、图片编辑和文生视频的开源框架。
  ☆243Updated this week
• 360AILAB-NLP / 360LayoutAnalysis
  360LayoutAnaylsis, a series Document Analysis Models and Datasets deleveped by 360 AI Research Institute
  ☆305Updated last year
• liguodongiot / unify-easy-llm
  unify-easy-llm（ULM）旨在打造一个简易的一键式大模型训练工具，支持Nvidia GPU、Ascend NPU等不同硬件以及常用的大模型。
  ☆59Updated last year
• CLUEbenchmark / SuperCLUE-RAG
  中文原生检索增强生成测评基准
  ☆124Updated last year
• jiangnanboy / llm_corpus_quality
  大模型预训练中文语料清洗及质量评估 Large model pre-training corpus cleaning
  ☆74Updated last year
• RapidAI / RapidLayout
  Analysis of Chinese and English layouts 中英文版面分析
  ☆260Updated 5 months ago
• shibing624 / agentica
  Agentica: Effortlessly Build Intelligent, Reflective, and Collaborative Multimodal AI Agents! 构建智能的多模态AI Agent。
  ☆230Updated last week
• kv1830 / fast_pdf_trans
  Fast pdf translate是一款pdf翻译软件，基于MinerU实现pdf转markdown的功能，接着对markdown进行分割， 送给大模型翻译，最后组装翻译结果并由pypandoc生成结果pdf。
  ☆39Updated 9 months ago
• riddle911 / SuperInsights
  ☆67Updated last year
• zjrwtx / SFT-data-builder
  利用免费的大模型api来结合你的私域数据来生成sft训练数据（妥妥白嫖）支持llamafactory等工具的训练数据格式synthetic data
  ☆191Updated last year
• thunlp / DeepNote
  ☆133Updated 9 months ago
• MetaGLM / LawGLM
  探索 LLM 在法律行业的应用潜力
  ☆98Updated last year
• tpoisonooo / ROGRAG
  [ACL2025 demo track] ROGRAG: A Robustly Optimized GraphRAG Framework
  ☆192Updated last month
• shibing624 / SearchGPT
  SearchGPT: Building a quick conversation-based search engine with LLMs.
  ☆46Updated last year
• li-xiu-qi / x-pdf2md
  本项目借助飞桨平台，构建起一套创新的多模型协同系统，实现 PDF 文件到 Markdown 文件的高效、精准转换。
  ☆27Updated 9 months ago
• Qihoo360 / 360zhinao
  360zhinao
  ☆291Updated 8 months ago
• KMnO4-zx / TinyAgent
  基于ReAct手搓一个Agent Demo
  ☆163Updated 6 months ago
• kaixindelele / ChatSensitiveWords
  利用LLM+敏感词库，来自动判别是否涉及敏感词。
  ☆136Updated 2 years ago
• linancn / TianGong-AI-Unstructure
  TianGong-AI-Unstructure
  ☆69Updated 3 months ago
• RapidAI / RapidDocEx
  📝 针对文档类图像做内容提取，将文档类图像一比一输出到Word或者Txt中，便于进一步使用或处理。后续计划支持输入PDF/图像，输出对应json格式、Txt格式、Word格式和Markdown格式。
  ☆207Updated last year
• hzauzxb / guidance-ocr
  视觉信息抽取任务中，使用OCR识别结果规范多模态大模型的回答
  ☆43Updated last year
• Alannikos / edg4llm
  A unified tool to generate fine-tuning datasets for LLMs, including questions, answers, and dialogues. ✨🤖📚💬
  ☆62Updated 10 months ago
• allwefantasy / byzer-llm
  Easy, fast, and cheap pretrain,finetune, serving for everyone
  ☆315Updated 6 months ago
• dataelement / bisheng-unstructured
  bisheng-unstructured library
  ☆57Updated 7 months ago
• billvsme / my_openai_api
  部署你自己的OpenAI api🤩, 基于flask, transformers (使用 Baichuan2-13B-Chat-4bits 模型, 可以运行在单张Tesla T4显卡) ，实现了OpenAI中Chat, Models和Completions接口，包含流式响…
  ☆96Updated 2 years ago
• aliyun / qwen-dianjin
  Qwen DianJin: LLMs for the Financial Industry by Alibaba Cloud（通义点金：阿里云金融大模型）
  ☆412Updated last week
• OKC13 / General-Documents-Layout-parser
  通用版面分析 | 中文文档解析 |Document Layout Analysis | layout paser
  ☆48Updated last year