dmis-lab / Monet
[ICLR 2025] Monet: Mixture of Monosemantic Experts for Transformers
☆60Updated 2 months ago
Alternatives and similar repositories for Monet:
Users that are interested in Monet are comparing it to the libraries listed below
- ☆74Updated 7 months ago
- The official repository for SkyLadder: Better and Faster Pretraining via Context Window Scheduling☆28Updated last week
- Pytorch implementation for "Compressed Context Memory For Online Language Model Interaction" (ICLR'24)☆55Updated 11 months ago
- Official repository of "LiNeS: Post-training Layer Scaling Prevents Forgetting and Enhances Model Merging"☆24Updated 4 months ago
- [𝐄𝐌𝐍𝐋𝐏 𝐅𝐢𝐧𝐝𝐢𝐧𝐠𝐬 𝟐𝟎𝟐𝟒 & 𝐀𝐂𝐋 𝟐𝟎𝟐𝟒 𝐍𝐋𝐑𝐒𝐄 𝐎𝐫𝐚𝐥] 𝘌𝘯𝘩𝘢𝘯𝘤𝘪𝘯𝘨 𝘔𝘢𝘵𝘩𝘦𝘮𝘢𝘵𝘪𝘤𝘢𝘭 𝘙𝘦𝘢𝘴𝘰𝘯𝘪𝘯…☆48Updated 10 months ago
- ☆41Updated 3 weeks ago
- ☆96Updated 9 months ago
- ☆49Updated 7 months ago
- Stanford NLP Python library for benchmarking the utility of LLM interpretability methods☆62Updated this week
- ☆76Updated 2 months ago
- Mask-Enhanced Autoregressive Prediction: Pay Less Attention to Learn More☆27Updated last month
- This repository contains the code used for the experiments in the paper "Fine-Tuning Enhances Existing Mechanisms: A Case Study on Entity…☆25Updated last year
- One Initialization to Rule them All: Fine-tuning via Explained Variance Adaptation☆38Updated 5 months ago
- A repository for research on medium sized language models.☆76Updated 10 months ago
- Exploration of automated dataset selection approaches at large scales.☆34Updated 3 weeks ago
- Anchored Preference Optimization and Contrastive Revisions: Addressing Underspecification in Alignment☆55Updated 7 months ago
- [ACL 2024] LangBridge: Multilingual Reasoning Without Multilingual Supervision☆87Updated 5 months ago
- ☆31Updated 2 months ago
- [NeurIPS 2024] Goldfish Loss: Mitigating Memorization in Generative LLMs☆84Updated 4 months ago
- Minimal implementation of the Self-Play Fine-Tuning Converts Weak Language Models to Strong Language Models paper (ArXiv 20232401.01335)☆29Updated last year
- Long Context Extension and Generalization in LLMs☆50Updated 6 months ago
- Code for ICLR 2025 Paper "What is Wrong with Perplexity for Long-context Language Modeling?"☆44Updated this week
- From GaLore to WeLore: How Low-Rank Weights Non-uniformly Emerge from Low-Rank Gradients. Ajay Jaiswal, Lu Yin, Zhenyu Zhang, Shiwei Liu,…☆44Updated 8 months ago
- Knowledge Unlearning for Large Language Models☆22Updated 3 weeks ago
- Multi-Layer Sparse Autoencoders (ICLR 2025)☆22Updated last month
- Open source replication of Anthropic's Crosscoders for Model Diffing☆48Updated 5 months ago
- Code for PHATGOOSE introduced in "Learning to Route Among Specialized Experts for Zero-Shot Generalization"☆82Updated last year
- Code for Zero-Shot Tokenizer Transfer☆125Updated 2 months ago
- Code for reproducing our paper "Not All Language Model Features Are Linear"☆73Updated 4 months ago
- Code for NeurIPS 2024 Spotlight: "Scaling Laws and Compute-Optimal Training Beyond Fixed Training Durations"☆71Updated 5 months ago