dmis-lab / Monet
[ICLR 2025] Monet: Mixture of Monosemantic Experts for Transformers
☆64Updated 3 months ago
Alternatives and similar repositories for Monet:
Users that are interested in Monet are comparing it to the libraries listed below
- ☆77Updated 8 months ago
- Stanford NLP Python library for benchmarking the utility of LLM interpretability methods☆70Updated 3 weeks ago
- This repository contains the code used for the experiments in the paper "Fine-Tuning Enhances Existing Mechanisms: A Case Study on Entity…☆25Updated last year
- [𝐄𝐌𝐍𝐋𝐏 𝐅𝐢𝐧𝐝𝐢𝐧𝐠𝐬 𝟐𝟎𝟐𝟒 & 𝐀𝐂𝐋 𝟐𝟎𝟐𝟒 𝐍𝐋𝐑𝐒𝐄 𝐎𝐫𝐚𝐥] 𝘌𝘯𝘩𝘢𝘯𝘤𝘪𝘯𝘨 𝘔𝘢𝘵𝘩𝘦𝘮𝘢𝘵𝘪𝘤𝘢𝘭 𝘙𝘦𝘢𝘴𝘰𝘯𝘪𝘯…☆49Updated 11 months ago
- Multi-Layer Sparse Autoencoders (ICLR 2025)☆22Updated 2 months ago
- The official repository for SkyLadder: Better and Faster Pretraining via Context Window Scheduling☆29Updated last month
- Code for reproducing our paper "Not All Language Model Features Are Linear"☆73Updated 4 months ago
- ☆36Updated last month
- Code for NeurIPS 2024 Spotlight: "Scaling Laws and Compute-Optimal Training Beyond Fixed Training Durations"☆71Updated 5 months ago
- Official implementation of "BERTs are Generative In-Context Learners"☆26Updated last month
- Official repository of "LiNeS: Post-training Layer Scaling Prevents Forgetting and Enhances Model Merging"☆25Updated 5 months ago
- Reference implementation for Reward-Augmented Decoding: Efficient Controlled Text Generation With a Unidirectional Reward Model☆43Updated last year
- Pytorch implementation for "Compressed Context Memory For Online Language Model Interaction" (ICLR'24)☆57Updated last year
- A repository for research on medium sized language models.☆76Updated 11 months ago
- [ACL 2024] LangBridge: Multilingual Reasoning Without Multilingual Supervision☆87Updated 5 months ago
- The official implementation of Self-Exploring Language Models (SELM)☆63Updated 10 months ago
- [NeurIPS 2024] Goldfish Loss: Mitigating Memorization in Generative LLMs☆84Updated 5 months ago
- Exploration of automated dataset selection approaches at large scales.☆38Updated last month
- ☆45Updated last month
- This repo is based on https://github.com/jiaweizzhao/GaLore☆26Updated 7 months ago
- PaCE: Parsimonious Concept Engineering for Large Language Models (NeurIPS 2024)☆35Updated 5 months ago
- Function Vectors in Large Language Models (ICLR 2024)☆161Updated this week
- Replicating O1 inference-time scaling laws☆83Updated 4 months ago
- Open source replication of Anthropic's Crosscoders for Model Diffing☆52Updated 5 months ago
- ☆51Updated 11 months ago
- From GaLore to WeLore: How Low-Rank Weights Non-uniformly Emerge from Low-Rank Gradients. Ajay Jaiswal, Lu Yin, Zhenyu Zhang, Shiwei Liu,…☆44Updated this week
- ☆96Updated 9 months ago
- ☆70Updated 5 months ago
- Simple and scalable tools for data-driven pretraining data selection.☆20Updated 2 months ago
- ☆50Updated 2 weeks ago