数据清洗，文本审核。DataFine mainly includes a number of data processing methods including rule cleaning, sensitive word filtering, advertisement filtering, de-duplication and sensitive content functions, providing safe and reliable data for the training of Chinese corpus.
☆12Feb 14, 2025Updated last year
Alternatives and similar repositories for DataFine
Users that are interested in DataFine are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- text security audit 安全审核-语义模型过滤 敏感内容检测系统☆40Feb 14, 2025Updated last year
- 利用分类法和敏感词检测法对生成式大模型的输入和输出内容进行安全检测，尽早识别风险内容。The input and output contents of generative large model are checked by classification method a…☆28Sep 9, 2024Updated 2 years ago
- 这里将paddle中的ocr等模型转为onnx格式，并利用java版深度框架djl加载这些onnx模型进行推理预测尝试。☆14Nov 15, 2022Updated 3 years ago
- TABLE DETECTION IN IMAGES AND OCR TO CSV WITH JAVA☆10Jul 18, 2023Updated 3 years ago
- t5-model-onnx，中文拼写纠错，Chinese spelling correction。☆15Sep 18, 2022Updated 4 years ago
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- textcnn for advertising detection，广告检测☆11Jan 12, 2024Updated 2 years ago
- model2onnx，将roberta和macbert模型转为onnx格式，并进行推理。☆19Jul 13, 2022Updated 4 years ago
- [CVPR 2025] An Implementation of the paper "Pre-Instruction Data Selection for Visual Instruction Tuning"☆17Jun 9, 2025Updated last year
- dsh-plugin-developer — DeepSeek Harness 插件开发 Skill > 指导 AI Agent 从 0 到 1 开发、构建、安装、测试 DeepSeek Harness（dsh）插件。基于 **dsh 0.1.1-rc.2** 与 dsh…☆31Sep 1, 2026Updated last month
- 深度网络实现意图分类。☆11Feb 26, 2021Updated 5 years ago
- MacBERT for Chinese Spelling Correction, macbert中文拼写纠错☆16May 23, 2022Updated 4 years ago
- IP限流平台，自动对异常访问的ＩＰ进行限制，会根据当前ＩＰ的异常访问情况，会按１分维度限制、１小时维度限制、２４小时维度限制及永久限制四个限制级别进行限制，并提供了可视化黑白名单管理配置、IP的QPS配置管理、按秒维度及分维度查看ＩＰ的即时访问等等功能。☆16Jul 5, 2021Updated 5 years ago
- 利用llm大语言模型提取卡证票据关键信息。Key Information Extraction from Image with LLM(large language model).Basically, it can extract key information from …☆15Jul 22, 2024Updated 2 years ago
- A Claude Code skill that turns Claude into an infinite development loop: spec → feature list → implement → test → commit → repeat — acros…☆13Mar 11, 2026Updated 7 months ago
- 1-Click AI Models by DigitalOcean Gradient • AdDeploy popular AI models on DigitalOcean Gradient GPU virtual machines with just a single click. Zero configuration with optimized deployments.
- Title and keywords are used to generate text.☆12Dec 6, 2021Updated 4 years ago
- 一个抖音评论爬虫程序，可以爬取指定关键词下的视频评论☆15Oct 17, 2024Updated 2 years ago
- 利用java对文章进行分析并图谱化展示（主要提取关键词、实体、依存分析等）。☆12Apr 14, 2023Updated 3 years ago
- 智能医疗诊断助手，Intelligent Medical Diagnosis Assistant☆20Nov 6, 2025Updated 11 months ago
- 大模型预训练中文语料清洗及质量评估 Large model pre-training corpus cleaning☆80Jul 25, 2024Updated 2 years ago
- ocr，pdf转docx，pdf to docx☆21Nov 4, 2022Updated 3 years ago
- albert-fc for RE(Relation Extraction)，中文关系抽取☆20Apr 24, 2023Updated 3 years ago
- 利用Swin-Unet(Swin Transformer Unet)实现对文档图片里表格结构的识别，Swin-unet (Swin Transformer Unet) is used to identify the document table structure☆27Feb 23, 2024Updated 2 years ago
- 利用sklearn和gensim中的tfidf,lsa,doc2vec进行查询与文档匹配搜索☆21Sep 11, 2022Updated 4 years ago
- GPU virtual machines on DigitalOcean Gradient AI • AdGet to production fast with high-performance AMD and NVIDIA GPUs you can spin up in seconds. The definition of operational simplicity.
- onnx-java，这里利用java加载onnx模型，并进行推理。☆22May 19, 2022Updated 4 years ago
- A curated list of excellent tools for data labeling and annotation across various domains.☆21May 28, 2025Updated last year
- albert-fc for LP(Link Prediction)，中文实体链接预测☆20Apr 21, 2023Updated 3 years ago
- ☆32Mar 25, 2026Updated 6 months ago
- 智能文本自动处理工具（Intelligent text automatic processing tool）。AutoText的功能主要有文本纠错，图片ocr、版面检测以及表格结构识别等。The main functions of this project include …☆28May 17, 2023Updated 3 years ago
- 利用java-yolov8实现版面检测（Chinese layout detection），java-yolov8 is used to detect the layout of Chinese document images☆27May 5, 2023Updated 3 years ago
- 地址解析，省+市+区+详细地址☆22Sep 5, 2023Updated 3 years ago
- AI 应用开发从入门到精通☆15Jun 21, 2025Updated last year
- Akcio is a demonstration project for Retrieval Augmented Generation (RAG). It leverages the power of LLM to generate responses and uses v…☆12Oct 30, 2023Updated 2 years ago
- AI Agents on DigitalOcean Gradient AI Platform • AdBuild production-ready AI agents using customizable tools or access multiple LLMs through a single endpoint. Create custom knowledge bases or connect external data.
- JFinalCMS，极速开发，动态添加字段，自定义标签，动态创建数据库表并crud数据，数据库备份、还原，动态添加站点(多站点功能)，一键生成模板代码，让您轻松打造自己的独立网站，同时也方便二次开发，让您快速搭建个性化独立网站，为您节约更多时间，去陪恋人、家人和朋友☆22Dec 16, 2023Updated 2 years ago
- 开源多品牌 RAW 照片编辑器，集调色、AI 增强、智能蒙版与照片合成于一体。 / An open-source multi-brand RAW photo editor with color grading, AI enhancement, smart masks, an…☆129Updated this week
- 题目知识点预测标注。Question knowledge point prediction.☆23Sep 11, 2022Updated 4 years ago
- chinese sentence punctuation prediction，中文句子标点符号预测。☆29Oct 19, 2022Updated 4 years ago
- gcn for prediction of protein interactions，图卷积用于蛋白质相互作用。☆24Nov 15, 2021Updated 4 years ago
- Codex Pro Max — Tauri v2 desktop launcher: taskboard service management, Codex CDP panel injection, ~/.codex config guard, FastCtx MCP in…☆40Oct 1, 2026Updated 2 weeks ago
- 工程控制论影响下的AIGC流程管理Skills集合。☆29Aug 13, 2026Updated 2 months ago