MingyuJ666 / Rope_with_LLMLinks
[ICML'25] Our study systematically investigates massive values in LLMs' attention mechanisms. First, we observe massive values are concentrated in low-frequency dimensions across different attention heads exclusively in attention queries (Q) and keys (K) while absent in values (V).
☆84Updated 6 months ago
Alternatives and similar repositories for Rope_with_LLM
Users that are interested in Rope_with_LLM are comparing it to the libraries listed below
Sorting:
- CoT-Valve: Length-Compressible Chain-of-Thought Tuning☆89Updated 11 months ago
- ☆60Updated 6 months ago
- Official repository for paper: O1-Pruner: Length-Harmonizing Fine-Tuning for O1-Like Reasoning Pruning☆97Updated 10 months ago
- A Sober Look at Language Model Reasoning☆92Updated 2 months ago
- [NeurIPS 2025] Think Silently, Think Fast: Dynamic Latent Compression of LLM Reasoning Chains☆69Updated 5 months ago
- [ICLR 2025] Code and Data Repo for Paper "Latent Space Chain-of-Embedding Enables Output-free LLM Self-Evaluation"☆93Updated last year
- [ICML 2024] Unveiling and Harnessing Hidden Attention Sinks: Enhancing Large Language Models without Training through Attention Calibrati…☆46Updated last year
- [EMNLP 2025] TokenSkip: Controllable Chain-of-Thought Compression in LLMs☆199Updated last month
- ☆204Updated 3 weeks ago
- 📜 Paper list on decoding methods for LLMs and LVLMs☆66Updated 2 months ago
- ☆73Updated 9 months ago
- This is the repository of DEER, a Dynamic Early Exit in Reasoning method for Large Reasoning Language Models.☆176Updated 6 months ago
- ☆140Updated 10 months ago
- The official repository of NeurIPS'25 paper "Ada-R1: From Long-Cot to Hybrid-CoT via Bi-Level Adaptive Reasoning Optimization"☆21Updated 2 months ago
- Chain of Thoughts (CoT) is so hot! so long! We need short reasoning process!☆72Updated 9 months ago
- One-shot Entropy Minimization☆187Updated 7 months ago
- ACL'2025: SoftCoT: Soft Chain-of-Thought for Efficient Reasoning with LLMs. and preprint: SoftCoT++: Test-Time Scaling with Soft Chain-of…☆74Updated 7 months ago
- ☆46Updated 3 months ago
- [NeurIPS 2025] The implementation of paper "On Reasoning Strength Planning in Large Reasoning Models"☆28Updated 6 months ago
- Resources and paper list for 'Scaling Environments for Agents'. This repository accompanies our survey on how environments contribute to …☆55Updated 3 weeks ago
- [NeurIPS 2025] Implementation for the paper "The Surprising Effectiveness of Negative Reinforcement in LLM Reasoning"☆152Updated 2 months ago
- ☆47Updated 9 months ago
- [TMLR 2025] Efficient Reasoning Models: A Survey☆292Updated 2 weeks ago
- [AAAI 2026] Official codebase for "GenPRM: Scaling Test-Time Compute of Process Reward Models via Generative Reasoning".☆94Updated 2 months ago
- [ICLR 2025] Dynamic Mixture of Experts: An Auto-Tuning Approach for Efficient Transformer Models☆153Updated 6 months ago
- dParallel: Learnable Parallel Decoding for dLLMs☆53Updated 3 months ago
- 🚀 LLaMA-MoE v2: Exploring Sparsity of LLaMA from Perspective of Mixture-of-Experts with Post-Training☆91Updated last year
- [EMNLP 2025] LightThinker: Thinking Step-by-Step Compression☆127Updated 9 months ago
- ☆348Updated 5 months ago
- ☆143Updated 4 months ago