UW-Madison-Lee-Lab / Expressive_Power_of_LoRALinks
Code for "The Expressive Power of Low-Rank Adaptation".
☆20Updated last year
Alternatives and similar repositories for Expressive_Power_of_LoRA
Users that are interested in Expressive_Power_of_LoRA are comparing it to the libraries listed below
Sorting:
- Official repository of paper "RNNs Are Not Transformers (Yet): The Key Bottleneck on In-context Retrieval"☆27Updated last year
- ☆32Updated 2 years ago
- ☆89Updated last year
- ☆29Updated 2 years ago
- ☆20Updated last year
- Code Release for "Broken Neural Scaling Laws" (BNSL) paper☆59Updated last year
- Curse-of-memory phenomenon of RNNs in sequence modelling☆19Updated 3 months ago
- ☆45Updated last year
- ☆18Updated 8 months ago
- Revisiting Efficient Training Algorithms For Transformer-based Language Models (NeurIPS 2023)☆80Updated last year
- Self-Supervised Alignment with Mutual Information☆21Updated last year
- The repository contains code for Adaptive Data Optimization☆25Updated 7 months ago
- Implementation of Influence Function approximations for differently sized ML models, using PyTorch☆15Updated last year
- Experiments and code to generate the GINC small-scale in-context learning dataset from "An Explanation for In-context Learning as Implici…☆108Updated last year
- ☆27Updated 5 months ago
- ☆12Updated 9 months ago
- Efficient Scaling laws and collaborative pretraining.☆16Updated 6 months ago
- ☆32Updated last year
- Official repository for our paper, Transformers Learn Higher-Order Optimization Methods for In-Context Learning: A Study with Linear Mode…☆17Updated 8 months ago
- Sparse and discrete interpretability tool for neural networks☆63Updated last year
- HGRN2: Gated Linear RNNs with State Expansion☆55Updated 11 months ago
- [ICLR 2023] "Sparse MoE as the New Dropout: Scaling Dense and Self-Slimmable Transformers" by Tianlong Chen*, Zhenyu Zhang*, Ajay Jaiswal…☆53Updated 2 years ago
- Lightweight Adapting for Black-Box Large Language Models☆23Updated last year
- Code for NeurIPS 2024 Spotlight: "Scaling Laws and Compute-Optimal Training Beyond Fixed Training Durations"☆81Updated 9 months ago
- Code for the paper "Data Feedback Loops: Model-driven Amplification of Dataset Biases"☆16Updated 2 years ago
- Exploration of automated dataset selection approaches at large scales.☆47Updated 5 months ago
- A Kernel-Based View of Language Model Fine-Tuning https://arxiv.org/abs/2210.05643☆78Updated last year
- This repository contains the code used for the experiments in the paper "Fine-Tuning Enhances Existing Mechanisms: A Case Study on Entity…☆27Updated last year
- Codebase for Context-aware Meta-learned Loss Scaling (CaMeLS). https://arxiv.org/abs/2305.15076.☆25Updated last year
- Companion repository to "Prompt Compression and Contrastive Conditioning for Controllability and Toxicity Reduction in Language Models"☆14Updated 2 years ago