mit-han-lab / offsite-tuningLinks
Offsite-Tuning: Transfer Learning without Full Model
☆379Updated last year
Alternatives and similar repositories for offsite-tuning
Users that are interested in offsite-tuning are comparing it to the libraries listed below
Sorting:
- ☆191Updated last year
- [COLM 2024] LoraHub: Efficient Cross-Task Generalization via Dynamic LoRA Composition☆652Updated last year
- Official code for our CVPR'22 paper “Vision Transformer Slimming: Multi-Dimension Searching in Continuous Optimization Space”☆250Updated last month
- A framework for merging models solving different tasks with different initializations into one multi-task model without any additional tr…☆308Updated last year
- Collection of Tools and Papers related to Adapters / Parameter-Efficient Transfer Learning/ Fine-Tuning☆198Updated last year
- Shepherd: A foundational framework enabling federated instruction tuning for large language models☆246Updated 2 years ago
- ☆230Updated last year
- A curated list of Model Merging methods.☆92Updated last year
- Official PyTorch implementation of QA-LoRA☆141Updated last year
- A simple and effective LLM pruning approach.☆808Updated last year
- ☆269Updated last year
- Code accompanying the paper "Massive Activations in Large Language Models"☆183Updated last year
- AdaLoRA: Adaptive Budget Allocation for Parameter-Efficient Fine-Tuning (ICLR 2023).☆352Updated 2 years ago
- Editing Models with Task Arithmetic☆504Updated last year
- DSIR large-scale data selection framework for language model training☆260Updated last year
- Code release for Dataless Knowledge Fusion by Merging Weights of Language Models (https://openreview.net/forum?id=FCnohuR6AnM)☆90Updated 2 years ago
- An Extensible Continual Learning Framework Focused on Language Models (LMs)☆286Updated last year
- This PyTorch package implements MoEBERT: from BERT to Mixture-of-Experts via Importance-Guided Adaptation (NAACL 2022).☆112Updated 3 years ago
- Official code for ReLoRA from the paper Stack More Layers Differently: High-Rank Training Through Low-Rank Updates☆464Updated last year
- [NeurIPS 2022] A Fast Post-Training Pruning Framework for Transformers☆192Updated 2 years ago
- ☆43Updated 11 months ago
- Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-models☆142Updated 3 years ago
- Official implementation of "DoRA: Weight-Decomposed Low-Rank Adaptation"☆124Updated last year
- Official PyTorch implementation of DistiLLM: Towards Streamlined Distillation for Large Language Models (ICML 2024)☆232Updated 7 months ago
- The official implementation of the paper "What Matters in Transformers? Not All Attention is Needed".☆177Updated 6 months ago
- Code for "SemDeDup", a simple method for identifying and removing semantic duplicates from a dataset (data pairs which are semantically s…☆144Updated 2 years ago
- Official repository of NEFTune: Noisy Embeddings Improves Instruction Finetuning☆400Updated last year
- Explorations into some recent techniques surrounding speculative decoding☆288Updated 9 months ago
- ICML'2022: Black-Box Tuning for Language-Model-as-a-Service & EMNLP'2022: BBTv2: Towards a Gradient-Free Future with Large Language Model…☆272Updated 2 years ago
- [ICLR 2023] "Learning to Grow Pretrained Models for Efficient Transformer Training" by Peihao Wang, Rameswar Panda, Lucas Torroba Hennige…☆92Updated last year