本项目旨在对大量文本文件进行快速编码检测和转换以辅助mnbvc语料集项目的数据清洗工作
☆71Sep 11, 2026Updated last month
Alternatives and similar repositories for charset_mnbvc
Users that are interested in charset_mnbvc are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- 文本去重☆76May 23, 2024Updated 2 years ago
- MNBVC项目-ShareGPT语料清洗☆17Oct 4, 2023Updated 3 years ago
- MNBVC(Massive Never-ending BT Vast Chinese corpus)超大规模中文语料集。对标chatGPT训练的40T数据。MNBVC数据集不但包括主流文化，也包括各个小众文化甚至火星文的数据。MNBVC数据集包括新闻、作文、小说、书籍、杂志…☆4,297Updated this week
- MNVBC项目裁判文书的清洗代码☆12Sep 11, 2025Updated last year
- implementation of https://arxiv.org/pdf/2312.09299☆21Jul 3, 2024Updated 2 years ago
- Simple, predictable pricing with DigitalOcean hosting • AdAlways know what you'll pay with monthly caps and flat pricing. Enterprise-grade infrastructure trusted by 600k+ customers.
- 更纯粹、更高压缩率的Tokenizer☆490Nov 27, 2024Updated last year
- Is a Openbuddy agent sysprompt. 这是一份Openbuddy智能体的系统提示词☆20Jun 16, 2026Updated 4 months ago
- Extract Chinese/English QA Data from WikiHow pages.☆17May 21, 2023Updated 3 years ago
- Code used for sourcing and cleaning the BigScience ROOTS corpus☆318Mar 20, 2023Updated 3 years ago
- 一个轻量级神经符号融合框架，通过结合小型语言模型与符号推理系统，在资源受限环境中实现高质量的逻辑推理能力。☆22Mar 2, 2025Updated last year
- parallel corpus dataset from the mnbvc project☆18Feb 11, 2026Updated 8 months ago
- ☆19May 11, 2024Updated 2 years ago
- Ongoing research training transformer language models at scale, including: BERT & GPT-2☆69Jul 20, 2023Updated 3 years ago
- 本项目主要对开源的MOSS SFT数据进行整理 ，转换成mnbvc多轮对话格式。MOSS-003涵盖用性、忠实性、无害性三个层面，共353w样本，MOSS-003 包含更细粒度的有用性类别标记、更广泛的无害性数据和更长对话轮数，共630w样本，☆13Dec 3, 2023Updated 2 years ago
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- ActionPiece: Rethinking Action Tokenization for Autoregressive Vision-Language-Action Models☆19Sep 17, 2026Updated last month
- ☆33Feb 9, 2025Updated last year
- Vocabulary Trimming (VT) is a model compression technique, which reduces a multilingual LM vocabulary to a target language by deleting ir…☆70Oct 25, 2024Updated 2 years ago
- The implementation of "Shallow-to-Deep Training for Neural Machine Translation"☆10Oct 26, 2020Updated 6 years ago
- 万能硬盘！☆32Nov 21, 2024Updated last year
- ☆33Nov 21, 2025Updated 11 months ago
- Asynchronous Python wrapper for the GPTZero API☆29Jan 9, 2024Updated 2 years ago
- A package containing utils for the PyTorch version of the Tapas algorithm.☆11Apr 29, 2021Updated 5 years ago
- A reproducible evaluation toolkit on spatial and embodied-intelligence benchmarks.☆26Sep 15, 2026Updated last month
- Wordpress hosting with auto-scaling - Free Trial Offer • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- A Simple MLLM Surpassed QwenVL-Max with OpenSource Data Only in 14B LLM.☆38Sep 9, 2024Updated 2 years ago
- calc total words from file☆11Sep 11, 2019Updated 7 years ago
- 这是一个一键让小参数大模型进行角色扮演的项目，从数据构成和训练都包含在这项目中☆27Mar 31, 2024Updated 2 years ago
- A more efficient GLM implementation!☆54Feb 18, 2023Updated 3 years ago
- INSET: Sentence Infilling with Inter-sentential Transformer☆30Nov 21, 2020Updated 5 years ago
- Implementation of DTMT with incremental decoding☆13Feb 20, 2021Updated 5 years ago
- ☆31Jun 5, 2025Updated last year
- [译] 自然语言处理综论 第三版☆26Jun 11, 2018Updated 8 years ago
- [EMNLP 2023] Official implementation of the algorithm ETSC: Exact Toeplitz-to-SSM Conversion our EMNLP 2023 paper - Accelerating Toeplitz…☆14Oct 17, 2023Updated 3 years ago
- GPUs on demand by Runpod - Special Offer Available • AdRun AI, ML, and HPC workloads on powerful cloud GPUs—without limits or wasted spend. Deploy GPUs in under a minute and pay by the second.
- ☆11Apr 10, 2023Updated 3 years ago
- Yoeman project generator to create an Electron app running a Flask server inside itself. Deployable via PyInstaller and Electron Forge pa…☆12Jun 29, 2021Updated 5 years ago
- Fine-tuned BERT on SQuAd 2.0 Dataset. Applied Knowledge Distillation (KD) and fine-tuned DistilBERT (student) using BERT as the teacher m…☆26Feb 13, 2021Updated 5 years ago
- Large-Scale High-quality Chinese Web Text with Multi-dimensional and fine-grained information☆41Dec 2, 2024Updated last year
- The official repository for Trust-Region Adaptive Policy Optimization (TRAPO) – a novel hybrid framework designed to enhance large langua…☆16Mar 2, 2026Updated 7 months ago
- 大模型预训练中文语料清洗及质量评估 Large model pre-training corpus cleaning☆80Jul 25, 2024Updated 2 years ago
- Toxic Comment Classification on Wikipedia Comments dataset.☆10May 5, 2019Updated 7 years ago