alvin-zyl / CoLA
Implementation of CoLA: Compute-Efficient Pre-Training of LLMs via Low-Rank Activation
☆16Updated last month
Alternatives and similar repositories for CoLA:
Users that are interested in CoLA are comparing it to the libraries listed below
- ☆50Updated last year
- ☆37Updated last year
- ☆93Updated last year
- official code for paper Probing the Decision Boundaries of In-context Learning in Large Language Models. https://arxiv.org/abs/2406.11233…☆16Updated 7 months ago
- A curated list of awesome resources dedicated to Scaling Laws for LLMs☆71Updated last year
- Representation Surgery for Multi-Task Model Merging. ICML, 2024.☆42Updated 5 months ago
- Source code of "Task arithmetic in the tangent space: Improved editing of pre-trained models".☆98Updated last year
- Bayesian low-rank adaptation for large language models☆23Updated 10 months ago
- ☆13Updated 10 months ago
- Preprint: Asymmetry in Low-Rank Adapters of Foundation Models☆35Updated last year
- Long Is More for Alignment: A Simple but Tough-to-Beat Baseline for Instruction Fine-Tuning [ICML 2024]☆17Updated 10 months ago
- Test-time-training on nearest neighbors for large language models☆39Updated 11 months ago
- A Mechanistic Understanding of Alignment Algorithms: A Case Study on DPO and Toxicity.☆71Updated 3 weeks ago
- ☆49Updated 7 months ago
- The code of “Improving Weak-to-Strong Generalization with Scalable Oversight and Ensemble Learning”☆16Updated last year
- Code for paper: Aligning Large Language Models with Representation Editing: A Control Perspective☆25Updated 2 months ago
- [NeurIPS 2023 Spotlight] Temperature Balancing, Layer-wise Weight Analysis, and Neural Network Training☆34Updated last year
- [NeurIPS 2024 Spotlight] EMR-Merging: Tuning-Free High-Performance Model Merging☆52Updated 3 weeks ago
- Easy-to-Hard Generalization: Scalable Alignment Beyond Human Supervision☆119Updated 6 months ago
- Official repository of "Localizing Task Information for Improved Model Merging and Compression" [ICML 2024]☆42Updated 5 months ago
- ☆66Updated 3 years ago
- [ICLR 2023] "Sparse MoE as the New Dropout: Scaling Dense and Self-Slimmable Transformers" by Tianlong Chen*, Zhenyu Zhang*, Ajay Jaiswal…☆48Updated 2 years ago
- Learning adapter weights from task descriptions☆16Updated last year
- Source code of ACL 2023 Main Conference Paper "PAD-Net: An Efficient Framework for Dynamic Networks".☆9Updated 11 months ago
- [NeurIPS 2023] Github repository for "Composing Parameter-Efficient Modules with Arithmetic Operations"☆60Updated last year
- [NeurIPS'24 Spotlight] Observational Scaling Laws☆54Updated 5 months ago
- A Kernel-Based View of Language Model Fine-Tuning https://arxiv.org/abs/2210.05643☆74Updated last year
- ☆73Updated last week
- This is the repository for "Model Merging by Uncertainty-Based Gradient Matching", ICLR 2024.☆27Updated 10 months ago
- ThinK: Thinner Key Cache by Query-Driven Pruning☆18Updated last month