Alternatives and similar repositories for massive-activations:

Users that are interested in massive-activations are comparing it to the libraries listed below

• prateeky2806 / ties-merging
  ☆165Updated last year
• gortizji / tangent_task_arithmetic
  Source code of "Task arithmetic in the tangent space: Improved editing of pre-trained models".
  ☆93Updated last year
• nightdessert / Retrieval_Head
  open-source code for paper: Retrieval Head Mechanistically Explains Long-Context Factuality
  ☆172Updated 6 months ago
• Lucky-Lance / Expert_Sparsity
  [ACL 2024] Not All Experts are Equal: Efficient Expert Pruning and Skipping for Mixture-of-Experts Large Language Models
  ☆81Updated 8 months ago
• ryoungj / ObsScaling
  [NeurIPS'24 Spotlight] Observational Scaling Laws
  ☆49Updated 4 months ago
• da03 / Internalize_CoT_Step_by_Step
  ☆146Updated last week
• UNITES-Lab / MC-SMoE
  [ICLR 2024 Spotlight] Code for the paper "Merge, Then Compress: Demystify Efficient SMoE with Hints from Its Routing Policy"
  ☆72Updated 8 months ago
• VITA-Group / Random-MoE-as-Dropout
  [ICLR 2023] "Sparse MoE as the New Dropout: Scaling Dense and Self-Slimmable Transformers" by Tianlong Chen*, Zhenyu Zhang*, Ajay Jaiswal…
  ☆48Updated last year
• roeehendel / icl_task_vectors
  ☆89Updated last year
• aim-uofa / LoRAPrune
  ☆46Updated 2 months ago
• SempraETY / Pruning-via-Merging
  ☆17Updated 2 months ago
• GATECH-EIC / ACT
  [ICML 2024] Unveiling and Harnessing Hidden Attention Sinks: Enhancing Large Language Models without Training through Attention Calibrati…
  ☆31Updated 7 months ago
• r-three / smear
  ☆27Updated last year
• nik-dim / tall_masks
  Official repository of "Localizing Task Information for Improved Model Merging and Compression" [ICML 2024]
  ☆39Updated 3 months ago
• princeton-nlp / ProLong
  Homepage for ProLong (Princeton long-context language models) and paper "How to Train Long-Context Language Models (Effectively)"
  ☆153Updated 2 months ago
• ericwtodd / function_vectors
  Function Vectors in Large Language Models (ICLR 2024)
  ☆137Updated 4 months ago
• LZY-the-boys / Twin-Merging
  [NeurIPS2024] Twin-Merging: Dynamic Integration of Modular Expertise in Model Merging
  ☆48Updated 2 months ago
• hkust-nlp / llm-compression-intelligence
  Official github repo for the paper "Compression Represents Intelligence Linearly" [COLM 2024]
  ☆130Updated 4 months ago
• logix-project / logix
  AI Logging for Interpretability and Explainability🔬
  ☆102Updated 8 months ago
• ScalingIntelligence / large_language_monkeys
  ☆82Updated 4 months ago
• ldery / Bonsai
  Code for "Everybody Prune Now: Structured Pruning of LLMs with only Forward Passes"
  ☆27Updated 10 months ago
• sail-sg / Attention-Sink
  [ICLR 2025] When Attention Sink Emerges in Language Models: An Empirical View (Spotlight)
  ☆48Updated 3 months ago
• sail-sg / scaling-with-vocab
  [NeurIPS-2024] 📈 Scaling Laws with Vocabulary: Larger Models Deserve Larger Vocabularies https://arxiv.org/abs/2407.13623
  ☆76Updated 4 months ago
• socialfoundations / tttlm
  Test-time-training on nearest neighbors for large language models
  ☆37Updated 9 months ago
• astramind-ai / Mixture-of-depths
  Unofficial implementation for the paper "Mixture-of-Depths: Dynamically allocating compute in transformer-based language models"
  ☆145Updated 7 months ago
• harveyhuang18 / EMR_Merging
  [NeurIPS 2024 Spotlight] EMR-Merging: Tuning-Free High-Performance Model Merging
  ☆44Updated 3 months ago
• HanGuo97 / lq-lora
  ☆125Updated last year
• lyy1994 / awesome-data-contamination
  The Paper List on Data Contamination for Large Language Models Evaluation.
  ☆90Updated last month
• lucidrains / coconut-pytorch
  Implementation of 🥥 Coconut, Chain of Continuous Thought, in Pytorch
  ☆152Updated last month
• microsoft / rho
  Repo for Rho-1: Token-level Data Selection & Selective Pretraining of LLMs.
  ☆396Updated 9 months ago