KaihuaTang / Qwen-Tokenizer-Pruner
Due to the huge vocaburary size (151,936) of Qwen models, the Embedding and LM Head weights are excessively heavy. Therefore, this project provides a Tokenizer vocabulary shearing solution for Qwen and Qwen-VL.
☆17Updated 7 months ago
Alternatives and similar repositories for Qwen-Tokenizer-Pruner:
Users that are interested in Qwen-Tokenizer-Pruner are comparing it to the libraries listed below
- TokenSkip: Controllable Chain-of-Thought Compression in LLMs☆103Updated 2 weeks ago
- [ICLR 2025] SWIFT: On-the-Fly Self-Speculative Decoding for LLM Inference Acceleration☆45Updated last month
- ☆72Updated last week
- This repo contains evaluation code for the paper "MileBench: Benchmarking MLLMs in Long Context"☆31Updated 8 months ago
- [ACL 2024] The official codebase for the paper "Self-Distillation Bridges Distribution Gap in Language Model Fine-tuning".☆115Updated 4 months ago
- The source code of "Merging Experts into One: Improving Computational Efficiency of Mixture of Experts (EMNLP 2023)":☆36Updated 11 months ago
- [ICLR 2025] PEARL: Parallel Speculative Decoding with Adaptive Draft Length☆67Updated 2 weeks ago
- Efficient Mixture of Experts for LLM Paper List☆51Updated 3 months ago
- [NeurIPS-2024] 📈 Scaling Laws with Vocabulary: Larger Models Deserve Larger Vocabularies https://arxiv.org/abs/2407.13623☆80Updated 6 months ago
- [AAAI 2024] Fluctuation-based Adaptive Structured Pruning for Large Language Models☆45Updated last year
- [ICLR 2024] CLEX: Continuous Length Extrapolation for Large Language Models☆76Updated last year
- Multi-Candidate Speculative Decoding☆35Updated 11 months ago
- ☆73Updated last week
- Code for paper "Patch-Level Training for Large Language Models"☆81Updated 4 months ago
- A prototype repo for hybrid training of pipeline parallel and distributed data parallel with comments on core code snippets. Feel free to…☆55Updated last year
- 🚀LLaMA-MoE v2: Exploring Sparsity of LLaMA from Perspective of Mixture-of-Experts with Post-Training☆75Updated 3 months ago
- Repository of LV-Eval Benchmark☆61Updated 7 months ago
- [ICML'24] The official implementation of “Rethinking Optimization and Architecture for Tiny Language Models”☆121Updated 2 months ago
- Repo for the EMNLP'24 Paper "Dual-Space Knowledge Distillation for Large Language Models". A general white-box KD framework for both same…☆44Updated 4 months ago
- [NeurIPS 2024] Fast Best-of-N Decoding via Speculative Rejection☆40Updated 5 months ago
- MLLM-Bench: Evaluating Multimodal LLMs with Per-sample Criteria☆66Updated 5 months ago
- qwen-nsa☆44Updated last week
- [ICLR 2025] 🧬 RegMix: Data Mixture as Regression for Language Model Pre-training (Spotlight)☆117Updated last month
- [ACL 2024 (Oral)] A Prospector of Long-Dependency Data for Large Language Models☆54Updated 8 months ago
- [ACL 2024] MT-Bench-101: A Fine-Grained Benchmark for Evaluating Large Language Models in Multi-Turn Dialogues☆78Updated 8 months ago
- [SIGIR'24] The official implementation code of MOELoRA.☆154Updated 8 months ago
- ☆98Updated 6 months ago
- [EMNLP 2024 Findings🔥] Official implementation of ": LOOK-M: Look-Once Optimization in KV Cache for Efficient Multimodal Long-Context In…☆92Updated 4 months ago
- [ICML'2024] Can AI Assistants Know What They Don't Know?☆79Updated last year
- ☆19Updated 2 weeks ago