maidacundo / MoE-LoRA
Adapt an LLM model to a Mixture-of-Experts model using Parameter Efficient finetuning (LoRA), injecting the LoRAs in the FFN.
☆21Updated 3 weeks ago
Related projects ⓘ
Alternatives and complementary repositories for MoE-LoRA
- [SIGIR'24] The official implementation code of MOELoRA.☆123Updated 3 months ago
- [ACL 2023] This is the code repo for our ACL'23 paper "Augmentation-Adapted Retriever Improves Generalization of Language Models as Gener…☆58Updated 3 months ago
- ToolEyes: Fine-Grained Evaluation for Tool Learning Capabilities of Large Language Models in Real-world Scenarios☆62Updated 6 months ago
- [ACL'24] Superfiltering: Weak-to-Strong Data Filtering for Fast Instruction-Tuning☆123Updated 2 months ago
- [ACL 2024] The official codebase for the paper "Self-Distillation Bridges Distribution Gap in Language Model Fine-tuning".☆96Updated last week
- [ICML'2024] Can AI Assistants Know What They Don't Know?☆70Updated 9 months ago
- ☆70Updated 10 months ago
- Scaling Sentence Embeddings with Large Language Models☆98Updated 7 months ago
- [EMNLP 2024] The official GitHub repo for the survey paper "Knowledge Conflicts for LLMs: A Survey"☆81Updated last month
- ☆51Updated 3 months ago
- ☆118Updated 6 months ago
- The code and data of DPA-RAG☆49Updated last month
- Paper list and datasets for the paper: A Survey on Data Selection for LLM Instruction Tuning☆32Updated 9 months ago
- Dataset and evaluation script for "Evaluating Hallucinations in Chinese Large Language Models"☆109Updated 5 months ago
- Counting-Stars (★)☆76Updated 2 months ago
- [EMNLP 2024 (Oral)] Leave No Document Behind: Benchmarking Long-Context LLMs with Extended Multi-Doc QA☆89Updated last month
- 🐋 An unofficial implementation of Self-Alignment with Instruction Backtranslation.☆131Updated 4 months ago
- [NAACL 2024 Outstanding Paper] Source code for the NAACL 2024 paper entitled "R-Tuning: Instructing Large Language Models to Say 'I Don't…☆84Updated 4 months ago
- [ICLR 2024] CLEX: Continuous Length Extrapolation for Large Language Models☆72Updated 7 months ago
- 🧬 RegMix: Data Mixture as Regression for Language Model Pre-training☆87Updated last month
- Do Large Language Models Know What They Don’t Know?☆85Updated this week
- Clustering and Ranking: Diversity-preserved Instruction Selection through Expert-aligned Quality Estimation☆65Updated last month
- [Neurips2024] Source code for xRAG: Extreme Context Compression for Retrieval-augmented Generation with One Token☆89Updated 4 months ago
- Implementation of the paper: "Making Retrieval-Augmented Language Models Robust to Irrelevant Context"☆61Updated 3 months ago
- Intuitive Fine-Tuning: Towards Simplifying Alignment into a Single Process☆22Updated 3 months ago
- self-adaptive in-context learning☆41Updated last year
- ☆37Updated 3 weeks ago
- [NAACL'24] Self-data filtering of LLM instruction-tuning data using a novel perplexity-based difficulty score, without using any other mo…☆305Updated 2 months ago
- Unofficial implementation of AlpaGasus☆84Updated last year
- Official repository for paper "Weak-to-Strong Extrapolation Expedites Alignment"☆67Updated 5 months ago