本项目旨在对大量文本文件进行快速编码检测和转换以辅助mnbvc语料集项目的数据清洗工作
☆71Sep 11, 2026Updated last week
Alternatives and similar repositories for charset_mnbvc
Users that are interested in charset_mnbvc are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- 文本去重☆76May 23, 2024Updated 2 years ago
- this repo is mnbvc text quality classification using fastText☆16Oct 2, 2023Updated 2 years ago
- MNBVC(Massive Never-ending BT Vast Chinese corpus)超大规模中文语料集。对标chatGPT训练的40T数据。MNBVC数据集不但包括主流文化，也包括各个小众文化甚至火星文的数据。MNBVC数据集包括新闻、作文、小说、书籍、杂志…☆4,275Updated this week
- Feature Decay Algorithms☆11Mar 5, 2014Updated 12 years ago
- ☆19May 11, 2024Updated 2 years ago
- Simple, predictable pricing with DigitalOcean hosting • AdAlways know what you'll pay with monthly caps and flat pricing. Enterprise-grade infrastructure trusted by 600k+ customers.
- Ongoing research training transformer language models at scale, including: BERT & GPT-2☆69Jul 20, 2023Updated 3 years ago
- A lightweight Python tool to analyze PCAP files and generate network traffic reports. It detects traffic patterns, security concerns, and…☆19Sep 25, 2024Updated last year
- RGCN model for real-time fraud detection☆11Jan 27, 2023Updated 3 years ago
- Transformer related optimization, including BERT, GPT☆39Feb 10, 2023Updated 3 years ago
- Implementing BERT + CRF with PyTorch for Chinese NER.☆10Mar 7, 2022Updated 4 years ago
- A package containing utils for the PyTorch version of the Tapas algorithm.☆11Apr 29, 2021Updated 5 years ago
- The implementation of "Shallow-to-Deep Training for Neural Machine Translation"☆10Oct 26, 2020Updated 5 years ago
- Code used for sourcing and cleaning the BigScience ROOTS corpus☆318Mar 20, 2023Updated 3 years ago
- Python implementation of AI-powered research assistant that performs iterative, deep research on any topic by combining search engines, w…☆50Mar 22, 2025Updated last year
- 1-Click AI Models by DigitalOcean Gradient • AdDeploy popular AI models on DigitalOcean Gradient GPU virtual machines with just a single click. Zero configuration with optimized deployments.
- 把教育信息化体系中的Word试题，Excel试卷、知识点等数据解析成json内容。☆14Mar 3, 2020Updated 6 years ago
- Codes and files for the paper Are Emergent Abilities in Large Language Models just In-Context Learning☆33Jan 9, 2025Updated last year
- 大模型预训练中文语料清洗及质量评估 Large model pre-training corpus cleaning☆80Jul 25, 2024Updated 2 years ago
- 更纯粹、更高压缩率的Tokenizer☆490Nov 27, 2024Updated last year
- Fine-tuned BERT on SQuAd 2.0 Dataset. Applied Knowledge Distillation (KD) and fine-tuned DistilBERT (student) using BERT as the teacher m…☆26Feb 13, 2021Updated 5 years ago
- A library to visualize algorithm by tracing your code.☆11May 31, 2026Updated 3 months ago
- ☆11Jul 10, 2022Updated 4 years ago
- TextPy: Collaborative Agent Workflow through Programming and Prompting☆27May 9, 2025Updated last year
- The official repository for Trust-Region Adaptive Policy Optimization (TRAPO) – a novel hybrid framework designed to enhance large langua…☆16Mar 2, 2026Updated 6 months ago
- AI Agents on DigitalOcean Gradient AI Platform • AdBuild production-ready AI agents using customizable tools or access multiple LLMs through a single endpoint. Create custom knowledge bases or connect external data.
- realize the reinforcement learning training for gpt2 llama bloom and so on llm model☆27Sep 19, 2023Updated 3 years ago
- Implementation of DTMT with incremental decoding☆13Feb 20, 2021Updated 5 years ago
- 复现 Soft-Masked BERT， 论文 Spelling Error Correction with Soft-Masked BERT☆12Oct 14, 2020Updated 5 years ago
- 数据管理平台（DataMan）是完全免费且开源的，任何人都可以无限制的修改代码以及部署服务，这对于很多想要对数据管理的应用平台来说是一个很好的选择：低廉的成本换回的是高效的管理方案，同时又有健康的生态提供支持。☆13Feb 25, 2022Updated 4 years ago
- A lightweight script for processing HTML page to markdown format with support for code blocks☆81Apr 14, 2024Updated 2 years ago
- https://marsvillager.github.io/PaperWorkflowAutomation/summary/visualize.html (Safari, Firefox ✓ Edge, Chrome ✗) 论文爬虫+概述: IEEE Symposium …☆12Oct 1, 2025Updated 11 months ago
- The code of ACL2022 paper "Conditional Bilingual Mutual Information based Adaptive Training for Neural Machine Translation"..☆14Aug 6, 2022Updated 4 years ago
- I-SHEEP: Iterative Self-enHancEmEnt Paradigm of LLMs through Self-Instruct and Self-Assessment☆17Jan 16, 2025Updated last year
- End-to-End Chinese Speaker Identification☆11Nov 17, 2022Updated 3 years ago
- GPUs on demand by Runpod - Special Offer Available • AdRun AI, ML, and HPC workloads on powerful cloud GPUs—without limits or wasted spend. Deploy GPUs in under a minute and pay by the second.
- ☆70Jun 28, 2022Updated 4 years ago
- All-in-one text de-duplication☆762Mar 9, 2026Updated 6 months ago
- 基于 Regularized Nash Dynamics 的军棋AI☆14Apr 2, 2025Updated last year
- ☆12Apr 7, 2026Updated 5 months ago
- C++ library for loading XDF files☆18Jul 20, 2026Updated last month
- Source codes of "Fast Continuous Subgraph Matching over Streaming Graphs via Backtracking Reduction", SIGMOD 2023☆14Sep 7, 2023Updated 3 years ago
- minimalist claude code☆16Aug 25, 2025Updated last year