sail-sg / Attention-Sink

Related projects ⓘ

Alternatives and complementary repositories for Attention-Sink

• tatsu-lab / test_set_contamination
  ☆33Updated last year
• SafeAILab / RAIN
  [ICLR'24] RAIN: Your Language Models Can Align Themselves without Finetuning
  ☆84Updated 5 months ago
• Leooyii / LCEG
  Long Context Extension and Generalization in LLMs
  ☆39Updated 2 months ago
• decoding-comp-trust / comp-trust
  Codebase for decoding compressed trust.
  ☆20Updated 6 months ago
• sail-sg / dice
  Official implementation of Bootstrapping Language Models via DPO Implicit Rewards
  ☆39Updated 3 months ago
• abhishekpanigrahi1996 / Skill-Localization-by-grafting
  ☆44Updated 10 months ago
• sail-sg / CPO
  [NeurIPS 2024] The official implementation of paper: Chain of Preference Optimization: Improving Chain-of-Thought Reasoning in LLMs.
  ☆63Updated last month
• sail-sg / Cheating-LLM-Benchmarks
  [SafeGenAi @ NeurIPS 2024] Cheating Automatic LLM Benchmarks: Null Models Achieve High Win Rates
  ☆60Updated 3 weeks ago
• GATECH-EIC / ACT
  [ICML 2024] Unveiling and Harnessing Hidden Attention Sinks: Enhancing Large Language Models without Training through Attention Calibrati…
  ☆25Updated 4 months ago
• RZFan525 / Awesome-ScalingLaws
  A curated list of awesome resources dedicated to Scaling Laws for LLMs
  ☆63Updated last year
• sail-sg / scaling-with-vocab
  [NeurIPS-2024] 📈 Scaling Laws with Vocabulary: Larger Models Deserve Larger Vocabularies https://arxiv.org/abs/2407.13623
  ☆69Updated last month
• zjunlp / KnowledgeCircuits
  [NeurIPS 2024] Knowledge Circuits in Pretrained Transformers
  ☆75Updated last month
• Unispac / shallow-vs-deep-alignment
  Official Repository for The Paper: Safety Alignment Should Be Made More Than Just a Few Tokens Deep
  ☆28Updated 4 months ago
• hkust-nlp / Activation_Decoding
  In-Context Sharpness as Alerts: An Inner Representation Perspective for Hallucination Mitigation (ICML 2024)
  ☆45Updated 7 months ago
• tml-epfl / long-is-more-for-alignment
  Long Is More for Alignment: A Simple but Tough-to-Beat Baseline for Instruction Fine-Tuning [ICML 2024]
  ☆15Updated 6 months ago
• nik-dim / tall_masks
  Official repository of "Localizing Task Information for Improved Model Merging and Compression" [ICML 2024]
  ☆35Updated 3 weeks ago
• Vaidehi99 / InfoDeletionAttacks
  ☆38Updated last year
• VITA-Group / Junk_DNA_Hypothesis
  [ICML 2024] Junk DNA Hypothesis: A Task-Centric Angle of LLM Pre-trained Weights through Sparsity; Lu Yin*, Ajay Jaiswal*, Shiwei Liu, So…
  ☆15Updated 5 months ago
• junkangwu / beta-DPO
  [NeurIPS 2024] Official code of $\beta$-DPO: Direct Preference Optimization with Dynamic $\beta$
  ☆29Updated 3 weeks ago
• SihengLi99 / LLM-Honesty-Survey
  A Survey on the Honesty of Large Language Models
  ☆46Updated last month
• EnnengYang / AdaMerging
  AdaMerging: Adaptive Model Merging for Multi-Task Learning. ICLR, 2024.
  ☆53Updated 3 weeks ago
• UNITES-Lab / MC-SMoE
  [ICLR 2024 Spotlight] Code for the paper "Merge, Then Compress: Demystify Efficient SMoE with Hints from Its Routing Policy"
  ☆64Updated 5 months ago
• shizhediao / Black-Box-Prompt-Learning
  Source code for the TMLR paper "Black-Box Prompt Learning for Pre-trained Language Models"
  ☆55Updated last year
• PKU-ML / LongPPL
  ☆17Updated this week
• 2003pro / ScaleBiO
  This is the official implementation of ScaleBiO: Scalable Bilevel Optimization for LLM Data Reweighting
  ☆14Updated 3 months ago
• facebookresearch / RLCD
  Reproduction of "RLCD Reinforcement Learning from Contrast Distillation for Language Model Alignment
  ☆65Updated last year
• waltonfuture / Diff-eRank
  Code for https://arxiv.org/abs/2401.17139 (NeurIPS 2024)
  ☆25Updated this week
• zhxieml / remiss-jailbreak
  ☆20Updated 4 months ago
• princeton-nlp / benign-data-breaks-safety
  ☆21Updated last month
• adymaharana / d2pruning
  ☆31Updated last year