Search & disable undertrain tokens in Qwen3.6 and other BPE tokenizer models to improve its performance in corner case (or more) / 寻找并禁用Qwen3.6(及其它BPE分词器)的欠训练token以提高corner case下（也许不止）的能力
☆29Apr 4, 2026Updated 4 months ago
Alternatives and similar repositories for qwen35_tokenizer_utils
Users that are interested in qwen35_tokenizer_utils are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Source Data of ACL2021 paper "Syntax-Enhanced Pre-trained Model"☆11Jun 1, 2021Updated 5 years ago
- ☆18Oct 8, 2023Updated 2 years ago
- Code repo for "Model-Generated Pretraining Signals Improves Zero-Shot Generalization of Text-to-Text Transformers" (ACL 2023)☆22Nov 1, 2023Updated 2 years ago
- {DeepL, Google, WMT-Best, davinci-003, turbo, gpt-4} × {En-De, En-Cs, En-Ru, En-Zh, De-Fr, En-Ja, Uk-En, Uk-Cs, En-Hr, En-Ha, En-Is}☆14Jun 18, 2023Updated 3 years ago
- Guidelines for our secondary layer of annotation adding multi-sentence AMR links☆12Sep 6, 2017Updated 8 years ago
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- 本程序的目的是将苹果手表过去的运动记录方便的上传至strava，并解决导出活动中心率字段缺失的问题，支持运动类型的自动判断。The purpose of this program is to easily upload past exercise records of Ap…☆15Nov 14, 2024Updated last year
- ☆13Aug 18, 2022Updated 3 years ago
- ☆35May 12, 2023Updated 3 years ago
- Code and Data for the ACL21 paper "Modeling Bilingual Conversational Characteristics for Neural Chat Translation"☆12Dec 17, 2021Updated 4 years ago
- ☆12May 23, 2022Updated 4 years ago
- Code repository for the ICML 2026 Oral paper "Characterizing, Evaluating, and Optimizing Complex Reasoning".☆18Jun 21, 2026Updated last month
- Study of Pre-Trained Positional Embeddings☆16Nov 6, 2020Updated 5 years ago
- ☆21Sep 5, 2023Updated 2 years ago
- Computational Social Science Lab introducing basic concepts, research design, and tools for text analysis.☆16Sep 13, 2024Updated last year
- Wordpress hosting with auto-scaling - Free Trial Offer • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- ☆15Feb 8, 2023Updated 3 years ago
- 《跨越网络的门槛》☆24Aug 7, 2022Updated 4 years ago
- Code release for "Generating Code World Models with Large Language Models Guided by Monte Carlo Tree Search" published at NeurIPS '24.☆20Feb 21, 2025Updated last year
- [ICLR 2026] The official implementation of the paper “Anchored Supervised Fine-Tuning”☆48Jun 19, 2026Updated last month
- ☆19May 15, 2026Updated 2 months ago
- "Knowledge Base Question Answering via Encoding of Complex Query Graphs" published in EMNLP 2018☆18Dec 31, 2020Updated 5 years ago
- A collection of instruction data and scripts for machine translation.☆20Sep 23, 2023Updated 2 years ago
- Official Pytorch Implementation of Paper "DarwinLM: Evolutionary Structured Pruning of Large Language Models"☆20Feb 21, 2025Updated last year
- ☆34Oct 2, 2024Updated last year
- 1-Click AI Models by DigitalOcean Gradient • AdDeploy popular AI models on DigitalOcean Gradient GPU virtual machines with just a single click. Zero configuration with optimized deployments.
- ☆27Jun 10, 2025Updated last year
- shadowsocks + v2ray-plugin☆16Jun 20, 2024Updated 2 years ago
- A Benchmark for Fine-Grained Relational Memory Discrimination in Long-Horizon AI Agents☆22Jun 9, 2026Updated 2 months ago
- Entropy-Driven GRPO with Guided Error Correction for Advantage Diversity☆22Aug 28, 2025Updated 11 months ago
- Code for "Think Natively: Unlocking Multilingual Reasoning with Consistency-Enhanced Reinforcement Learning".☆27Nov 11, 2025Updated 9 months ago
- Code for our work "MSP: Multi-Stage Prompting for Making Pre-trained Language Models Better Translators" in ACL 2022☆20Mar 18, 2022Updated 4 years ago
- A simple syllabus view for Android.☆27Sep 19, 2016Updated 9 years ago
- Some visualization with different python packages☆25Aug 10, 2019Updated 7 years ago
- ☆34Jan 17, 2025Updated last year
- Wordpress hosting with auto-scaling - Free Trial Offer • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- ☆43May 22, 2025Updated last year
- [NeurIPS 2024] Official code of $\beta$-DPO: Direct Preference Optimization with Dynamic $\beta$☆51Oct 23, 2024Updated last year
- Code for SIGIR-2020 full paper: DukeNet: A Dual Knowledge Interaction Network for Knowledge-Grounded Conversation☆29Feb 1, 2021Updated 5 years ago
- An Agent-native Obsidian wiki + CLAUDE.md skeleton that lets Agent know your project as well as you do.☆29Apr 5, 2026Updated 4 months ago
- ☆35Feb 12, 2026Updated 6 months ago
- This is AI implementation (not official) of the DreamGym framework from the paper "Scaling Agent Learning via Experience Synthesis" (arXi…☆46Nov 9, 2025Updated 9 months ago
- ☆55Feb 11, 2025Updated last year