sail-sg / Attention-Sink

Related projects ⓘ

Alternatives and complementary repositories for Attention-Sink

• sail-sg / dice
  Official implementation of Bootstrapping Language Models via DPO Implicit Rewards
  ☆39Updated 3 months ago
• tatsu-lab / test_set_contamination
  ☆33Updated last year
• yegcjs / DiffusionLLM
  Code for paper "Diffusion Language Models Can Perform Many Tasks with Scaling and Instruction-Finetuning"
  ☆63Updated 9 months ago
• sail-sg / Cheating-LLM-Benchmarks
  [SafeGenAi @ NeurIPS 2024] Cheating Automatic LLM Benchmarks: Null Models Achieve High Win Rates
  ☆59Updated 2 weeks ago
• RZFan525 / Awesome-ScalingLaws
  A curated list of awesome resources dedicated to Scaling Laws for LLMs
  ☆63Updated last year
• r-three / smear
  ☆27Updated last year
• SafeAILab / RAIN
  [ICLR'24] RAIN: Your Language Models Can Align Themselves without Finetuning
  ☆83Updated 5 months ago
• GATECH-EIC / ACT
  [ICML 2024] Unveiling and Harnessing Hidden Attention Sinks: Enhancing Large Language Models without Training through Attention Calibrati…
  ☆21Updated 4 months ago
• PKU-ML / LongPPL
  ☆15Updated last week
• HKUNLP / DiffuLLaMA
  DiffuGPT and DiffuLLaMA: Scaling Diffusion Language Models via Adaptation from Autoregressive Models
  ☆56Updated 2 weeks ago
• tml-epfl / long-is-more-for-alignment
  Long Is More for Alignment: A Simple but Tough-to-Beat Baseline for Instruction Fine-Tuning [ICML 2024]
  ☆14Updated 6 months ago
• decoding-comp-trust / comp-trust
  Codebase for decoding compressed trust.
  ☆20Updated 6 months ago
• sail-sg / D-TRAK
  Intriguing Properties of Data Attribution on Diffusion Models (ICLR 2024)
  ☆23Updated 9 months ago
• zjunlp / KnowledgeCircuits
  [NeurIPS 2024] Knowledge Circuits in Pretrained Transformers
  ☆68Updated 3 weeks ago
• princeton-nlp / benign-data-breaks-safety
  ☆19Updated last month
• adymaharana / d2pruning
  ☆31Updated last year
• Leooyii / LCEG
  Long Context Extension and Generalization in LLMs
  ☆39Updated last month
• nik-dim / tall_masks
  Official repository of "Localizing Task Information for Improved Model Merging and Compression" [ICML 2024]
  ☆34Updated 2 weeks ago
• sail-sg / scaling-with-vocab
  [NeurIPS-2024] 📈 Scaling Laws with Vocabulary: Larger Models Deserve Larger Vocabularies https://arxiv.org/abs/2407.13623
  ☆67Updated last month
• junkangwu / beta-DPO
  [NeurIPS 2024] Official code of $\beta$-DPO: Direct Preference Optimization with Dynamic $\beta$
  ☆29Updated 2 weeks ago
• JasonForJoy / Model-Editing-Hurt
  EMNLP 2024: Model Editing Harms General Abilities of Large Language Models: Regularization to the Rescue
  ☆32Updated 3 weeks ago
• abhishekpanigrahi1996 / Skill-Localization-by-grafting
  ☆44Updated 10 months ago
• Shwai-He / MEO
  The source code of "Merging Experts into One: Improving Computational Efficiency of Mixture of Experts (EMNLP 2023)":
  ☆34Updated 7 months ago
• Unispac / shallow-vs-deep-alignment
  Official Repository for The Paper: Safety Alignment Should Be Made More Than Just a Few Tokens Deep
  ☆26Updated 4 months ago
• LZY-the-boys / Twin-Merging
  [NeurIPS2024] Twin-Merging: Dynamic Integration of Modular Expertise in Model Merging
  ☆30Updated 3 weeks ago
• cathyxl / MAgIC
  ☆34Updated 10 months ago
• zhxieml / remiss-jailbreak
  ☆20Updated 4 months ago
• VITA-Group / Junk_DNA_Hypothesis
  [ICML 2024] Junk DNA Hypothesis: A Task-Centric Angle of LLM Pre-trained Weights through Sparsity; Lu Yin*, Ajay Jaiswal*, Shiwei Liu, So…
  ☆15Updated 5 months ago
• SihengLi99 / LLM-Honesty-Survey
  A Survey on the Honesty of Large Language Models
  ☆44Updated last month
• hkust-nlp / PEM_composition
  [NeurIPS 2023] Github repository for "Composing Parameter-Efficient Modules with Arithmetic Operations"
  ☆58Updated 11 months ago