EricLBuehler / xloraLinks
X-LoRA: Mixture of LoRA Experts
☆236Updated last year
Alternatives and similar repositories for xlora
Users that are interested in xlora are comparing it to the libraries listed below
Sorting:
- Parameter-Efficient Sparsity Crafting From Dense to Mixture-of-Experts for Instruction Tuning on General Tasks (EMNLP'24)☆147Updated 11 months ago
- Repo for Rho-1: Token-level Data Selection & Selective Pretraining of LLMs.☆431Updated last year
- ☆269Updated last year
- Implementation of CALM from the paper "LLM Augmented LLMs: Expanding Capabilities through Composition", out of Google Deepmind☆177Updated 11 months ago
- Unofficial implementation for the paper "Mixture-of-Depths: Dynamically allocating compute in transformer-based language models"☆169Updated last year
- LongRoPE is a novel method that can extends the context window of pre-trained LLMs to an impressive 2048k tokens.☆241Updated last year
- [NeurIPS 2024] Official Repository of The Mamba in the Llama: Distilling and Accelerating Hybrid Models☆226Updated 3 months ago
- The official implementation of the paper "What Matters in Transformers? Not All Attention is Needed".☆174Updated 4 months ago
- Code for "Critique Fine-Tuning: Learning to Critique is More Effective than Learning to Imitate" [COLM 2025]☆170Updated last month
- ☆187Updated 4 months ago
- The Truth Is In There: Improving Reasoning in Language Models with Layer-Selective Rank Reduction☆388Updated last year
- Implementation of 🥥 Coconut, Chain of Continuous Thought, in Pytorch☆179Updated 2 months ago
- [ACL'24] Selective Reflection-Tuning: Student-Selected Data Recycling for LLM Instruction-Tuning☆360Updated 11 months ago
- Benchmarking LLMs with Challenging Tasks from Real Users☆237Updated 9 months ago
- ☆202Updated 8 months ago
- Code for NeurIPS'24 paper 'Grokked Transformers are Implicit Reasoners: A Mechanistic Journey to the Edge of Generalization'☆229Updated last month
- Official repository for ORPO☆462Updated last year
- ☆187Updated last year
- Block Transformer: Global-to-Local Language Modeling for Fast Inference (NeurIPS 2024)☆160Updated 4 months ago
- The official repo for "LLoCo: Learning Long Contexts Offline"☆118Updated last year
- ☆221Updated last year
- Spherical Merge Pytorch/HF format Language Models with minimal feature loss.☆135Updated last year
- Positional Skip-wise Training for Efficient Context Window Extension of LLMs to Extremely Length (ICLR 2024)☆206Updated last year
- This is the official repository for Inheritune.☆112Updated 6 months ago
- ☆86Updated 7 months ago
- Unofficial Implementation of Evolutionary Model Merging☆39Updated last year
- Implementation of DoRA☆301Updated last year
- The HELMET Benchmark☆167Updated last week
- This is the official repository for the paper "Flora: Low-Rank Adapters Are Secretly Gradient Compressors" in ICML 2024.☆104Updated last year
- AnchorAttention: Improved attention for LLMs long-context training☆212Updated 7 months ago