codes and plots for "Active-Dormant Attention Heads: Mechanistically Demystifying Extreme-Token Phenomena in LLMs"
☆11Dec 30, 2024Updated last year
Alternatives and similar repositories for Active-Dormant-Attention
Users that are interested in Active-Dormant-Attention are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Code to generate figures of paper "When do spectral gradient updates help in deep learning?"☆19Dec 3, 2025Updated 11 months ago
- Example code of Sparse Gaussian Process Attention (ICLR 2023)☆26Sep 15, 2025Updated last year
- Code for "Matching Features, Not Tokens: Energy-Based Fine-Tuning of Language Models".☆23Mar 16, 2026Updated 7 months ago
- Visualization of mean field and neural tangent kernel regime☆23Jul 25, 2024Updated 2 years ago
- Unofficial Implementation of Selective Attention Transformer☆20Oct 31, 2024Updated 2 years ago
- Proton VPN Special Offer - Get 70% off • AdSpecial partner offer. Trusted by over 100 million users worldwide. Tested, Approved and Recommended by Experts.
- Qwen3 MoE Router Statistics | 큐웬3 MoE☆11May 16, 2025Updated last year
- Multi-Layer Sparse Autoencoders (ICLR 2025)☆31Oct 11, 2026Updated 3 weeks ago
- A Playground for Variational Autoencoders☆12Feb 11, 2018Updated 8 years ago
- Code for Augment & Reduce, a scalable stochastic algorithm for large categorical distributions☆10May 16, 2018Updated 8 years ago
- Dataset and pre-trained model of EMNLP-IJCNLP 2019 paper "TalkDown: A Corpus for Condescension Detection in Context."☆10Jan 26, 2020Updated 6 years ago
- ☆10Jul 27, 2026Updated 3 months ago
- ☆14Jul 13, 2025Updated last year
- Dynamical low rank training for neural networks☆11Aug 24, 2023Updated 3 years ago
- Medusa: Accelerating Serverless LLM Inference with Materialization [ASPLOS'25]☆12Nov 8, 2024Updated 2 years ago
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- ☆10Aug 26, 2022Updated 4 years ago
- Code for "Exponential Family Estimation via Adversarial Dynamics Embedding" (NeurIPS 2019)☆14Nov 26, 2019Updated 6 years ago
- Jupyter notebooks from our weekly (or so) hackathons☆11Dec 3, 2024Updated last year
- ☆12Jul 6, 2021Updated 5 years ago
- ☆16May 24, 2026Updated 5 months ago
- ☆17Sep 24, 2026Updated last month
- Official Pytorch implementation of Chromatic Graph Transformers☆10Jun 14, 2023Updated 3 years ago
- Experiments on the impact of depth in transformers and SSMs.☆48Oct 23, 2025Updated last year
- Clustered Compositional Embeddings☆13Oct 25, 2023Updated 3 years ago
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- Samplers from the paper "Stochastic Gradient MCMC with Repulsive Forces"☆11Feb 3, 2020Updated 6 years ago
- [ICLR'26] Beyond Linear Probes: Dynamic Safety Monitoring for Language Models☆13Oct 1, 2025Updated last year
- Fast, Numerically Stable, and Auto-Differentiable Spectral Clipping via Newton-Schulz Iteration☆12Jun 21, 2025Updated last year
- ☆13Apr 16, 2024Updated 2 years ago
- ☆38Feb 8, 2026Updated 8 months ago
- ☆16Apr 26, 2023Updated 3 years ago
- PyTorch utilities for ML, specifically speech☆13Jan 30, 2024Updated 2 years ago
- Code accompanying VarGrad: A Low-Variance Gradient Estimator for Variational Inference☆12Oct 12, 2020Updated 6 years ago
- Least Squares Regression for subspace clustering☆11May 27, 2018Updated 8 years ago
- Wordpress hosting with auto-scaling - Free Trial Offer • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- my type shit☆21Aug 3, 2026Updated 3 months ago
- This is the official code repository for the paper NeurIPS 2024 spotlight paper "Kermut: Composite kernel regression for protein variant …☆48Aug 5, 2025Updated last year
- Posterior with interesting shapes from actually used models☆13Feb 10, 2025Updated last year
- An implementation of Raft Consensus Algorithm in Elixir☆21Apr 3, 2016Updated 10 years ago
- [COLM 2026] An efficient 3D sampling method for long-CoT LLM.☆16May 25, 2025Updated last year
- This is the official implementation for our ACL 2024 paper: "Causal Estimation of Memorisation Profiles".☆25Mar 25, 2025Updated last year
- Ἀνατομή is a PyTorch library to analyze representation of neural networks☆13Jan 31, 2024Updated 2 years ago