Ablustrund / LoRAMoE
LoRAMoE: Revolutionizing Mixture of Experts for Maintaining World Knowledge in Language Model Alignment
☆231Updated 6 months ago
Related projects ⓘ
Alternatives and complementary repositories for LoRAMoE
- ☆116Updated 4 months ago
- ☆147Updated 4 months ago
- [SIGIR'24] The official implementation code of MOELoRA.☆127Updated 4 months ago
- ☆155Updated last month
- AdaLoRA: Adaptive Budget Allocation for Parameter-Efficient Fine-Tuning (ICLR 2023).☆278Updated last year
- [NAACL'24] Self-data filtering of LLM instruction-tuning data using a novel perplexity-based difficulty score, without using any other mo…☆308Updated 2 months ago
- Reading notes about Multimodal Large Language Models, Large Language Models, and Diffusion Models☆154Updated 2 weeks ago
- Implementation for "Step-DPO: Step-wise Preference Optimization for Long-chain Reasoning of LLMs"☆287Updated 4 months ago
- ☆77Updated 4 months ago
- [ACL 2024] The official codebase for the paper "Self-Distillation Bridges Distribution Gap in Language Model Fine-tuning".☆100Updated 3 weeks ago
- [CVPR'24] RLHF-V: Towards Trustworthy MLLMs via Behavior Alignment from Fine-grained Correctional Human Feedback☆236Updated 2 months ago
- InsTag: A Tool for Data Analysis in LLM Supervised Fine-tuning☆218Updated last year
- Repository for Label Words are Anchors: An Information Flow Perspective for Understanding In-Context Learning☆153Updated 9 months ago
- ☆71Updated 10 months ago
- ☆120Updated 7 months ago
- [ICML 2024] LESS: Selecting Influential Data for Targeted Instruction Tuning☆376Updated last month
- A generalized framework for subspace tuning methods in parameter efficient fine-tuning.☆104Updated 2 months ago
- Paper List for In-context Learning 🌷☆171Updated 9 months ago
- Must-read Papers on Large Language Model (LLM) Continual Learning☆134Updated last year
- ☆247Updated last year
- A RLHF Infrastructure for Vision-Language Models☆106Updated last week
- [EMNLP 2024] LongAlign: A Recipe for Long Context Alignment of LLMs☆221Updated 7 months ago
- Deita: Data-Efficient Instruction Tuning for Alignment [ICLR2024]☆500Updated 6 months ago
- Code for "Lion: Adversarial Distillation of Proprietary Large Language Models (EMNLP 2023)"☆201Updated 9 months ago
- The related works and background techniques about Openai o1☆144Updated 2 weeks ago
- Continual Learning of Large Language Models: A Comprehensive Survey☆255Updated last week
- State-of-the-art Parameter-Efficient MoE Fine-tuning Method☆93Updated 3 months ago
- A repository sharing the literatures about long-context large language models, including the methodologies and the evaluation benchmarks☆252Updated 3 months ago
- MoCLE (First MLLM with MoE for instruction customization and generalization!) (https://arxiv.org/abs/2312.12379)☆29Updated 7 months ago
- [ACL'24] Superfiltering: Weak-to-Strong Data Filtering for Fast Instruction-Tuning☆126Updated 2 months ago