大模型预训练中文语料清洗及质量评估 Large model pre-training corpus cleaning
☆80Jul 25, 2024Updated 2 years ago
Alternatives and similar repositories for llm_corpus_quality
Users that are interested in llm_corpus_quality are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- textcnn for advertising detection，广告检测☆11Jan 12, 2024Updated 2 years ago
- text security audit 安全审核-语义模型过滤 敏感内容检测系统☆39Feb 14, 2025Updated last year
- 利用分类法和敏感词检测法对生成式大模型的输入和输出内容进行安全检测，尽早识别风险内容。The input and output contents of generative large model are checked by classification method a…☆28Sep 9, 2024Updated last year
- chinese sentence punctuation prediction，中文句子标点符号预测。☆29Oct 19, 2022Updated 3 years ago
- 基于simhash的文本去重算法☆20Jun 18, 2021Updated 5 years ago
- Managed Kubernetes at scale on DigitalOcean • AdDigitalOcean Kubernetes includes the control plane, bandwidth allowance, container registry, automatic updates, and more for free.
- A comprehensive toolkit for streamlining data editing, search, and inspection for large-scale language model training and interpretabilit…☆21Oct 30, 2025Updated 9 months ago
- onnx-java，这里利用java加载onnx模型，并进行推理。☆21May 19, 2022Updated 4 years ago
- 利用llm大语言模型提取卡证票据关键信息。Key Information Extraction from Image with LLM(large language model).Basically, it can extract key information from …☆15Jul 22, 2024Updated 2 years ago
- spark tutorial for big data mining。包括app流量运营分析、als推荐、smote样本采样、RFM客户价值分群、AHP层次分析客户价值得分、手机定位数据商圈挖掘、马尔可夫智能邮件预测、时序预测、关联规则、推荐电影好友等。☆40Sep 10, 2022Updated 3 years ago
- MacBERT for Chinese Spelling Correction, macbert中文拼写纠错☆16May 23, 2022Updated 4 years ago
- Build a simple basic multimodal large model from scratch. 从零搭建一个简单的基础多模态大模型🤖☆48Jun 19, 2024Updated 2 years ago
- near-synonym, 基于大模型LLM的中文反义词/近义词(antonyms/synonyms)工具包. 也可计算词语相似度/句子相似度/文本相似度等。☆31Apr 29, 2025Updated last year
- Title and keywords are used to generate text.☆12Dec 6, 2021Updated 4 years ago
- Spring Deep Java Library 通过利用DJL框架与其他Spring框架进行整合，进行深度学习模型训练和推导。☆24Mar 16, 2022Updated 4 years ago
- 1-Click AI Models by DigitalOcean Gradient • AdDeploy popular AI models on DigitalOcean Gradient GPU virtual machines with just a single click. Zero configuration with optimized deployments.
- Python Scritpt which can be embedded into PyTorch model to print the model size.☆19Apr 19, 2021Updated 5 years ago
- 利用java对文章进行分析并图谱化展示（主要提取关键词、实体、依存分析等）。☆12Apr 14, 2023Updated 3 years ago
- Code & Data for our Paper "NaSGEC: Multi-Domain Chinese Grammatical Error Correction for Native Speaker Texts" (ACL 2023 Findings)☆96Feb 18, 2025Updated last year
- ☆366Jun 13, 2024Updated 2 years ago
- 供AI训练的中文数据集（持续更新。。。）与AI公司图谱，目前的数据集餐饮行业8000问，百度知道，Alpaca中文数据集，计算机领域数据集，Vicuna数据集，RedPajama数据集，Wikipedia中文词条数据集，网站论坛问答数据集☆66Nov 29, 2023Updated 2 years ago
- 中文版面检测（Chinese layout detection），yolov8 is used to detect the layout of Chinese document images。☆61Apr 28, 2023Updated 3 years ago
- 智能医疗诊断助手，Intelligent Medical Diagnosis Assistant☆20Nov 6, 2025Updated 9 months ago
- ☆22Dec 8, 2022Updated 3 years ago
- 本项目旨在对大量文本文件进行快速编码检测和转换以辅助mnbvc语料集项目的数据清洗工作☆70Oct 17, 2025Updated 9 months ago
- AI Agents on DigitalOcean Gradient AI Platform • AdBuild production-ready AI agents using customizable tools or access multiple LLMs through a single endpoint. Create custom knowledge bases or connect external data.
- Official PyTorch implementation for "Where You Edit is What You Get: Text-Guided Image Editing with Region-Based Attention" (Pattern Reco…☆10Oct 1, 2024Updated last year
- Python implementation of Lloyd-Max quantizer.☆23Apr 4, 2021Updated 5 years ago
- albert-fc for RE(Relation Extraction)，中文关系抽取☆20Apr 24, 2023Updated 3 years ago
- model2onnx，将roberta和macbert模型转为onnx格式，并进行推理。☆19Jul 13, 2022Updated 4 years ago
- 用于生成文本纠错模型(如Gector)需要的大量数据。☆15Jan 5, 2023Updated 3 years ago
- 利用sklearn和gensim中的tfidf,lsa,doc2vec进行查询与文档匹配搜索☆21Sep 11, 2022Updated 3 years ago
- albert-fc for LP(Link Prediction)，中文实体链接预测☆19Apr 21, 2023Updated 3 years ago
- This repository open-sources our GEC system submitted by THU KELab (sz) in the CCL2023-CLTC Track 1: Multidimensional Chinese Learner Tex…☆15Nov 25, 2023Updated 2 years ago
- The Corpus & Code for EMNLP 2022 paper "FCGEC: Fine-Grained Corpus for Chinese Grammatical Error Correction" | FCGEC中文语法纠错语料及STG模型☆123Apr 12, 2026Updated 3 months ago
- Proton VPN Special Offer - Get 70% off • AdSpecial partner offer. Trusted by over 100 million users worldwide. Tested, Approved and Recommended by Experts.
- 对llava官方代码的一些学习笔记☆29Oct 11, 2024Updated last year
- Build Neo4J Knowledge Graphs from Excel files☆23Nov 18, 2024Updated last year
- jcorrector 中文文本纠错工具， Text Error Correction Tool，Spelling Check☆82Mar 2, 2026Updated 5 months ago
- 数据基本清洗包括日期、时间、数值、字符串、字符、金钱、数据库（mysql、postgresql、mongodb、hbase、hdfsmemcached）、加解密（md5、sha、base64、aes、rsa）、文件、http服务、正则表达式等，后期会不断更新。☆13Jul 25, 2018Updated 8 years ago
- Efficiently computing & storing token n-grams from large corpora☆28Jun 15, 2026Updated last month
- 天池比赛☆10Jul 4, 2021Updated 5 years ago
- Towards Real-World Writing Assistance: A Chinese Character Checking Benchmark with Faked and Misspelled Characters☆17May 30, 2024Updated 2 years ago