KaihuaTang / Qwen-Tokenizer-PrunerView external linksLinks
Due to the huge vocaburary size (151,936) of Qwen models, the Embedding and LM Head weights are excessively heavy. Therefore, this project provides a Tokenizer vocabulary shearing solution for Qwen and Qwen-VL.
☆32Jan 6, 2026Updated last month
Alternatives and similar repositories for Qwen-Tokenizer-Pruner
Users that are interested in Qwen-Tokenizer-Pruner are comparing it to the libraries listed below
Sorting:
- 本项目提供了基于910B的huggingface LLM模型的Tensor Parallel(TP)部署教程，同时也可以作为一份极简的TP学习代码。☆30Jan 6, 2026Updated last month
- GPTQ inference TVM kernel☆40Apr 25, 2024Updated last year
- Conic10K: A large-scale dataset for closed-vocabulary math problem understanding. Accepted to EMNLP2023 Findings.☆31Dec 6, 2023Updated 2 years ago
- My Implementation of Q-Sparse: All Large Language Models can be Fully Sparsely-Activated☆33Aug 14, 2024Updated last year
- InSales e-commerce platform API bindings☆14Jul 13, 2024Updated last year
- ☆30Dec 27, 2024Updated last year
- Official implementation of "Automated Generation of Challenging Multiple-Choice Questions for Vision Language Model Evaluation" (CVPR 202…☆40May 26, 2025Updated 8 months ago
- ☆44Nov 1, 2025Updated 3 months ago
- ☆11Feb 19, 2022Updated 3 years ago
- [NeurIPS 2023] Generalized Logit Adjustment☆39Apr 21, 2024Updated last year
- ☆13Nov 5, 2024Updated last year
- [ACM MM 2023] The released code of paper "Deconfounded Visual Question Generation with Causal Inference"☆11Sep 3, 2024Updated last year
- Nanos klib for NVIDIA GPUs☆14Mar 25, 2025Updated 10 months ago
- Implementation of a fast semantic chunker in C++, installable in python 3.7+ projects.☆22Sep 20, 2025Updated 4 months ago
- A tool for generating synthetic data.☆19Dec 19, 2025Updated last month
- A local, voice-controlled AI assistant with the personality of HAL 9000 from 2001: A Space Odyssey.☆20Aug 16, 2025Updated 6 months ago
- ☆29Dec 20, 2025Updated last month
- Simple repository for training small reasoning models☆49Feb 6, 2025Updated last year
- A unified programming framework for high and portable performance across FPGAs and GPUs☆11Mar 23, 2025Updated 10 months ago
- Piper based VoiceDock TTS implementation☆11Aug 12, 2023Updated 2 years ago
- ☆10Dec 26, 2023Updated 2 years ago
- ☆15Feb 11, 2025Updated last year
- Tokenizer for Text to Speech (TTS) models☆13Jan 16, 2025Updated last year
- ☆10Jul 18, 2023Updated 2 years ago
- This repository contains the dataset of the paper ARGUS: Context-Based Detection of Stealthy IoT Infiltration Attacks☆12Apr 28, 2023Updated 2 years ago
- [ICML 2025 Oral] This is the official repository of the paper "What Limits Virtual Agent Application? OmniBench: A Scalable Multi-Dimensi…☆20Jun 12, 2025Updated 8 months ago
- a simple programming language under development☆11Dec 3, 2023Updated 2 years ago
- A transparent glass style for qt applications.☆32Jan 26, 2026Updated 3 weeks ago
- Package for word stress detection☆11Jan 27, 2023Updated 3 years ago
- A hackable library for running and fine-tuning modern transformer models on commodity and alternative GPUs, powered by tinygrad.☆28Feb 10, 2026Updated last week
- Automaton & Cognition☆16Apr 14, 2024Updated last year
- 使用自然语言绘制流程图，基于OpenAI☆12Nov 13, 2023Updated 2 years ago
- Pdf Query chat-bot using Gemini AI and Llma Index☆10Dec 24, 2023Updated 2 years ago
- gym_fetch_env with insert drawer open door☆13Mar 22, 2022Updated 3 years ago
- Official repository for the paper, "FedMABench: Benchmarking Mobile GUI Agents on Decentralized Heterogeneous User Data", EMNLP 2025 Main…☆15Nov 11, 2025Updated 3 months ago
- [AAAI 2025] Assessing the Creativity of LLMs in Proposing Novel Solutions to Mathematical Problems☆12May 5, 2025Updated 9 months ago
- Model-based Hindsight Experience Replay☆10Jun 8, 2022Updated 3 years ago
- [CIKM 2022] Towards Automated Over-Sampling for Imbalanced Classification☆10Mar 20, 2023Updated 2 years ago
- paper code commit-fsmafl☆10Mar 18, 2024Updated last year