Alternatives and similar repositories for regex-tokenizer

Users that are interested in regex-tokenizer are comparing it to the libraries listed below

• wangyuxinwhy / generate
  A Python Package to Access World-Class Generative Models
  ☆127Updated last year
• intsig-textin / markdown_tester
  如需体验textin文档解析，请点击https://cc.co/16YSIy
  ☆112Updated 3 weeks ago
• SmartFlowAI / Hand-on-RAG
  顾名思义：手搓的RAG
  ☆125Updated last year
• zjrwtx / SFT-data-builder
  利用免费的大模型api来结合你的私域数据来生成sft训练数据（妥妥白嫖）支持llamafactory等工具的训练数据格式synthetic data
  ☆169Updated 7 months ago
• shibing624 / agentica
  Agentica: Effortlessly Build Intelligent, Reflective, and Collaborative Multimodal AI Agents! 构建智能的多模态AI Agent。
  ☆187Updated last week
• CLUEbenchmark / SuperCLUE-RAG
  中文原生检索增强生成测评基准
  ☆119Updated last year
• riddle911 / SuperInsights
  ☆66Updated 10 months ago
• li-xiu-qi / x-pdf2md
  本项目借助飞桨平台，构建起一套创新的多模型协同系统，实现 PDF 文件到 Markdown 文件的高效、精准转换。
  ☆20Updated 3 months ago
• 360AILAB-NLP / 360LayoutAnalysis
  360LayoutAnaylsis, a series Document Analysis Models and Datasets deleveped by 360 AI Research Institute
  ☆292Updated 10 months ago
• RapidAI / RapidLayout
  Analysis of Chinese and English layouts 中英文版面分析
  ☆226Updated this week
• shell-nlp / gpt_server
  gpt_server是一个用于生产级部署LLMs、Embedding、Reranker、ASR和TTS的开源框架。
  ☆200Updated last week
• jasonkylelol / graphrag-chinese
  支持中文🇨🇳🇨🇳🇨🇳 的 microsoft/graphrag
  ☆48Updated 3 months ago
• liguodongiot / unify-easy-llm
  unify-easy-llm（ULM）旨在打造一个简易的一键式大模型训练工具，支持Nvidia GPU、Ascend NPU等不同硬件以及常用的大模型。
  ☆56Updated 11 months ago
• zzlgreat / smart_agent
  ☆105Updated last year
• SWHL / ChineseDocumentPDF
  中文论文、证券类、财报类PDF数据
  ☆33Updated last year
• kv1830 / fast_pdf_trans
  Fast pdf translate是一款pdf翻译软件，基于MinerU实现pdf转markdown的功能，接着对markdown进行分割， 送给大模型翻译，最后组装翻译结果并由pypandoc生成结果pdf。
  ☆25Updated 3 months ago
• aigc-apps / PAI-RAG
  An easy-to-use framework for modular RAG
  ☆375Updated this week
• hzauzxb / guidance-ocr
  视觉信息抽取任务中，使用OCR识别结果规范多模态大模型的回答
  ☆37Updated 6 months ago
• MetaGLM / LawGLM
  探索 LLM 在法律行业的应用潜力
  ☆90Updated 7 months ago
• jiangnanboy / llm_corpus_quality
  大模型预训练中文语料清洗及质量评估 Large model pre-training corpus cleaning
  ☆67Updated 11 months ago
• wey-gu / grpo-graph-extraction
  Qwen GRPO Graph Extraction RL Finetune
  ☆51Updated 3 months ago
• RapidAI / RapidOCRPDF
  Based on RapidOCR, extract the PDF content
  ☆174Updated 2 months ago
• tianchiguaixia / ocr_recognition
  微调阿里开源的文字检测模型，利用合合识别返回的OCR结果作为初始训练数据，对模型进行优化训练，使其更加适应1万张图片的具体场景，提高文字识别的精度。
  ☆9Updated 7 months ago
• ictnlp / FlexRAG
  FlexRAG: A RAG Framework for Information Retrieval and Generation.
  ☆194Updated last month
• DocAILab / XRAG
  XRAG: eXamining the Core - Benchmarking Foundational Component Modules in Advanced Retrieval-Augmented Generation
  ☆103Updated 2 weeks ago
• Qihoo360 / 360zhinao
  360zhinao
  ☆290Updated 2 months ago
• hyperai / vllm-cn
  vLLM Documentation in Chinese Simplified / vLLM 中文文档
  ☆85Updated 2 months ago
• infinigence / InfiniWebSearch
  A demo built on Megrez-3B-Instruct, integrating a web search tool to enhance the model's question-and-answer capabilities.
  ☆38Updated 7 months ago
• Alibaba-NLP / CHRONOS
  Repo for NAACL 2025 Paper "Unfolding the Headline: Iterative Self-Questioning for News Retrieval and Timeline Summarization"
  ☆281Updated 5 months ago
• RapidAI / RapidDoc
  📝 针对文档类图像做内容提取，将文档类图像一比一输出到Word或者Txt中，便于进一步使用或处理。后续计划支持输入PDF/图像，输出对应json格式、Txt格式、Word格式和Markdown格式。
  ☆198Updated 8 months ago