Alternatives and similar repositories for attention

Users that are interested in attention are comparing it to the libraries listed below

• allenai / wimbd
  What's In My Big Data (WIMBD) - a toolkit for analyzing large text datasets
  ☆218Updated 6 months ago
• justinchiu / openlogprobs
  Extract full next-token probabilities via language model APIs
  ☆244Updated last year
• nelson-liu / lost-in-the-middle
  Code and data for "Lost in the Middle: How Language Models Use Long Contexts"
  ☆343Updated last year
• felipemaiapolo / tinyBenchmarks
  Evaluating LLMs with fewer examples
  ☆153Updated last year
• shengliu66 / ICV
  Code for In-context Vectors: Making In Context Learning More Effective and Controllable Through Latent Space Steering
  ☆172Updated 3 months ago
• lukasberglund / reversal_curse
  ☆288Updated last year
• andyrdt / refusal_direction
  Code and results accompanying the paper "Refusal in Language Models Is Mediated by a Single Direction".
  ☆217Updated 7 months ago
• msclar / formatspread
  Code accompanying "How I learned to start worrying about prompt formatting".
  ☆105Updated 7 months ago
• nrimsky / CAA
  Steering Llama 2 with Contrastive Activation Addition
  ☆151Updated 11 months ago
• da03 / Internalize_CoT_Step_by_Step
  ☆171Updated 3 weeks ago
• ericwtodd / function_vectors
  Function Vectors in Large Language Models (ICLR 2024)
  ☆166Updated last month
• kmeng01 / memit
  Mass-editing thousands of facts into a transformer memory (ICLR 2023)
  ☆490Updated last year
• EleutherAI / delphi
  Delphi was the home of a temple to Phoebus Apollo, which famously had the inscription, 'Know Thyself.' This library lets language models …
  ☆172Updated this week
• nrimsky / LM-exp
  LLM experiments done during SERI MATS - focusing on activation steering / interpreting activation spaces
  ☆92Updated last year
• callummcdougall / sae_vis
  Create feature-centric and prompt-centric visualizations for sparse autoencoders (like those from Anthropic's published research).
  ☆201Updated 5 months ago
• meg-tong / sycophancy-eval
  datasets from the paper "Towards Understanding Sycophancy in Language Models"
  ☆75Updated last year
• OSU-NLP-Group / GrokkedTransformer
  Code for NeurIPS'24 paper 'Grokked Transformers are Implicit Reasoners: A Mechanistic Journey to the Edge of Generalization'
  ☆191Updated 5 months ago
• steering-vectors / steering-vectors
  Steering vectors for transformer language models in Pytorch / Huggingface
  ☆100Updated 2 months ago
• likenneth / honest_llama
  Inference-Time Intervention: Eliciting Truthful Answers from a Language Model
  ☆522Updated 3 months ago
• saprmarks / geometry-of-truth
  ☆82Updated 9 months ago
• allenai / WildBench
  Benchmarking LLMs with Challenging Tasks from Real Users
  ☆222Updated 6 months ago
• dwzhu-pku / LongEmbed
  LongEmbed: Extending Embedding Models for Long Context Retrieval (EMNLP 2024)
  ☆135Updated 6 months ago
• huggingface / cosmopedia
  ☆515Updated 5 months ago
• MadryLab / context-cite
  Attribute (or cite) statements generated by LLMs back to in-context information.
  ☆231Updated 7 months ago
• collin-burns / discovering_latent_knowledge
  ☆265Updated last year
• jacobdunefsky / transcoder_circuits
  ☆112Updated 6 months ago
• princeton-nlp / AutoCompressors
  [EMNLP 2023] Adapting Language Models to Compress Long Contexts
  ☆304Updated 8 months ago
• google / sycophancy-intervention
  Scripts for generating synthetic finetuning data for reducing sycophancy.
  ☆111Updated last year
• microsoft / rho
  Repo for Rho-1: Token-level Data Selection & Selective Pretraining of LLMs.
  ☆412Updated last year
• mcleish7 / arithmetic
  Code to reproduce "Transformers Can Do Arithmetic with the Right Embeddings", McLeish et al (NeurIPS 2024)
  ☆189Updated 11 months ago