Alternatives and similar repositories for Attention-Sink

Users that are interested in Attention-Sink are comparing it to the libraries listed below

• bethgelab / sober-reasoning
  A Sober Look at Language Model Reasoning
  ☆52Updated this week
• GATECH-EIC / ACT
  [ICML 2024] Unveiling and Harnessing Hidden Attention Sinks: Enhancing Large Language Models without Training through Attention Calibrati…
  ☆38Updated 11 months ago
• sail-sg / AnytimeReasoner
  Optimizing Anytime Reasoning via Budget Relative Policy Optimization
  ☆36Updated last week
• ryoungj / BoLT
  Code for "Reasoning to Learn from Latent Thoughts"
  ☆104Updated 2 months ago
• PKU-ML / LongPPL
  Code for ICLR 2025 Paper "What is Wrong with Perplexity for Long-context Language Modeling?"
  ☆78Updated 2 weeks ago
• Dereck0602 / Awesome_Test_Time_LLMs
  ☆105Updated 2 months ago
• OpenSparseLLMs / MoM
  ☆83Updated last month
• holarissun / RewardModelingBeyondBradleyTerry
  official implementation of ICLR'2025 paper: Rethinking Bradley-Terry Models in Preference-based Reward Modeling: Foundations, Theory, and…
  ☆58Updated 2 months ago
• Joshua-Ren / Learning_dynamics_LLM
  ☆131Updated 2 weeks ago
• sail-sg / dice
  Official implementation of Bootstrapping Language Models via DPO Implicit Rewards
  ☆44Updated last month
• VITA-Group / SEAL
  Official code for SEAL: Steerable Reasoning Calibration of Large Language Models for Free
  ☆25Updated last month
• Wang-ML-Lab / multimodal-needle-in-a-haystack
  [NAACL 2025 Oral] Multimodal Needle in a Haystack (MMNeedle): Benchmarking Long-Context Capability of Multimodal Large Language Models
  ☆44Updated last month
• hkust-nlp / mstar
  [ICML 2025] M-STAR (Multimodal Self-Evolving TrAining for Reasoning) Project. Diving into Self-Evolving Training for Multimodal Reasoning
  ☆60Updated 5 months ago
• ZhentingWang / DUMP
  ☆19Updated 3 weeks ago
• MingyuJ666 / Rope_with_LLM
  [ICML'25] Our study systematically investigates massive values in LLMs' attention mechanisms. First, we observe massive values are concen…
  ☆66Updated last week
• sail-sg / ActivePRM
  ☆15Updated last month
• javiferran / sae_entities
  ☆36Updated 2 months ago
• horseee / CoT-Valve
  CoT-Valve: Length-Compressible Chain-of-Thought Tuning
  ☆69Updated 3 months ago
• harveyhuang18 / EMR_Merging
  [NeurIPS 2024 Spotlight] EMR-Merging: Tuning-Free High-Performance Model Merging
  ☆59Updated 3 months ago
• Dongping-Chen / MLLM-Judge
  [ICML 2024 Oral] Official code repository for MLLM-as-a-Judge.
  ☆68Updated 3 months ago
• facebookresearch / iGSM
  The code for creating the iGSM datasets in papers "Physics of Language Models Part 2.1, Grade-School Math and the Hidden Reasoning Proces…
  ☆54Updated 4 months ago
• kamanphoebe / Look-into-MoEs
  [NAACL 2025] A Closer Look into Mixture-of-Experts in Large Language Models
  ☆52Updated 3 months ago
• ruixin31 / Rethink_RLVR
  ☆231Updated last week
• PRIME-RL / Entropy-Mechanism-of-RL
  The Entropy Mechanism of Reinforcement Learning for Large Language Model Reasoning.
  ☆69Updated this week
• Edward-Sun / easy-to-hard
  Easy-to-Hard Generalization: Scalable Alignment Beyond Human Supervision
  ☆120Updated 8 months ago
• thu-wyz / inference_scaling
  ☆69Updated 6 months ago
• luka-group / vlm-knowledge-conflict
  Code for paper "Unraveling Cross-Modality Knowledge Conflicts in Large Vision-Language Models."
  ☆42Updated 7 months ago
• tmlr-group / landscape-of-thoughts
  [ICLR 2025 Workshop] "Landscape of Thoughts: Visualizing the Reasoning Process of Large Language Models"
  ☆19Updated last week
• tatsu-lab / test_set_contamination
  ☆37Updated last year
• RLHFlow / Directional-Preference-Alignment
  Directional Preference Alignment
  ☆56Updated 8 months ago