Search & disable undertrain tokens in Qwen3.6 and other BPE tokenizer models to improve its performance in corner case (or more) / 寻找并禁用Qwen3.6(及其它BPE分词器)的欠训练token以提高corner case下（也许不止）的能力
☆28Apr 4, 2026Updated 6 months ago
Alternatives and similar repositories for qwen35_tokenizer_utils
Users that are interested in qwen35_tokenizer_utils are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Official codebase for the Scattered Forest Search: Smarter Code Space Exploration and Inference Scaling with LLMs☆11Feb 20, 2025Updated last year
- This project research aims to address the issue of overly long thinking chains in the Qwen3.5 model through data distillation-based fine-…☆24Mar 25, 2026Updated 7 months ago
- Guidelines for our secondary layer of annotation adding multi-sentence AMR links☆12Sep 6, 2017Updated 9 years ago
- realize the reinforcement learning training for gpt2 llama bloom and so on llm model☆27Sep 19, 2023Updated 3 years ago
- ☆15Apr 11, 2024Updated 2 years ago
- Managed Database hosting by DigitalOcean • AdPostgreSQL, MySQL, MongoDB, Kafka, Valkey, and OpenSearch available. Automatically scale up storage and focus on building your apps.
- 本程序的目的是将苹果手表过去的运动记录方便的上传至strava，并解决导出活动中心率字段缺失的问题，支持运动类型的自动判断。The purpose of this program is to easily upload past exercise records of Ap…☆15Nov 14, 2024Updated last year
- {DeepL, Google, WMT-Best, davinci-003, turbo, gpt-4} × {En-De, En-Cs, En-Ru, En-Zh, De-Fr, En-Ja, Uk-En, Uk-Cs, En-Hr, En-Ha, En-Is}☆14Jun 18, 2023Updated 3 years ago
- 💵 Code for Less is More for Long Document Summary Evaluation by LLMs (Wu*, Iso* et al; EACL 2024)☆12Feb 22, 2024Updated 2 years ago
- Code and data realeases for the paper -- TasTe: Teaching Large Language Models to Translate through Self-Reflection☆12Aug 17, 2024Updated 2 years ago
- ☆14Oct 6, 2025Updated last year
- [KDD 2026 ADS Track] Pytorch implementation of the paper "Hi-Guard: Towards Trustworthy Multimodal Moderation via Policy-Aligned Reasonin…☆27Jan 13, 2026Updated 9 months ago
- P1-VL: Bridging Visual Perception and Scientific Reasoning in Physics Olympiads☆15Feb 11, 2026Updated 8 months ago
- Study of Pre-Trained Positional Embeddings☆16Nov 6, 2020Updated 5 years ago
- Code release for "Generating Code World Models with Large Language Models Guided by Monte Carlo Tree Search" published at NeurIPS '24.☆20Feb 21, 2025Updated last year
- End-to-end encrypted cloud storage - Proton Drive • AdSpecial offer: 40% Off Yearly / 80% Off First Month. Protect your most important files, photos, and documents from prying eyes.
- Computational Social Science Lab introducing basic concepts, research design, and tools for text analysis.☆16Sep 13, 2024Updated 2 years ago
- ☆15Feb 8, 2023Updated 3 years ago
- Dialogue Graph Modeling for Conversational Machine Reading (ACL 2021, Findings)☆18Nov 29, 2022Updated 3 years ago
- "Knowledge Base Question Answering via Encoding of Complex Query Graphs" published in EMNLP 2018☆18Dec 31, 2020Updated 5 years ago
- A collection of instruction data and scripts for machine translation.☆20Sep 23, 2023Updated 3 years ago
- TreeRL: LLM Reinforcement Learning with On-Policy Tree Search in ACL'25☆106Jun 16, 2025Updated last year
- [ICLR 2026] The official implementation of the paper “Anchored Supervised Fine-Tuning”☆51Sep 8, 2026Updated last month
- 本项目使用云问科技训练的中文版UniLM模型对微博数据集进行自动标题生成。☆38Apr 4, 2024Updated 2 years ago
- ☆22Feb 4, 2026Updated 8 months ago
- Proton VPN Special Offer - Get 70% off • AdSpecial partner offer. Trusted by over 100 million users worldwide. Tested, Approved and Recommended by Experts.
- ☆23Sep 9, 2026Updated last month
- ☆34Oct 2, 2024Updated 2 years ago
- A 4B diffusion grounding model for fast, precise visual evidence extraction. Unifies diverse grounding tasks through parallel decoding, w…☆103Oct 14, 2026Updated last week
- ☆21Oct 18, 2026Updated last week
- ☆30Aug 29, 2026Updated last month
- A simple syllabus view for Android.☆27Sep 19, 2016Updated 10 years ago
- Data and code for the paper "Future is not One-dimensional: Complex Event Schema Induction via Graph Modeling".☆30Apr 24, 2021Updated 5 years ago
- Code for our work "MSP: Multi-Stage Prompting for Making Pre-trained Language Models Better Translators" in ACL 2022☆20Mar 18, 2022Updated 4 years ago
- [EMNLP' 26 Findings] The official code repository for SubtleMemory: A Benchmark for Fine-Grained Relational Memory Discrimination in Long…☆23Oct 5, 2026Updated 3 weeks ago
- Simple, predictable pricing with DigitalOcean hosting • AdAlways know what you'll pay with monthly caps and flat pricing. Enterprise-grade infrastructure trusted by 600k+ customers.
- ☆42May 22, 2025Updated last year
- This repository is a sub branch of AI Knowledge Tree, mainly focus on Natural Language Processing.☆27Jun 14, 2021Updated 5 years ago
- VMProtect Web端打包&序列号生成工具。☆14Oct 6, 2026Updated 3 weeks ago
- The original Shared Recurrent Memory Transformer implementation☆39Aug 24, 2026Updated 2 months ago
- Code for SIGIR-2020 full paper: DukeNet: A Dual Knowledge Interaction Network for Knowledge-Grounded Conversation☆29Feb 1, 2021Updated 5 years ago
- A torch model analyzer☆10Jul 24, 2025Updated last year
- [NeurIPS 2024] Official code of $\beta$-DPO: Direct Preference Optimization with Dynamic $\beta$☆52Oct 23, 2024Updated 2 years ago