EricLBuehler / xlora
X-LoRA: Mixture of LoRA Experts
☆215Updated 8 months ago
Alternatives and similar repositories for xlora:
Users that are interested in xlora are comparing it to the libraries listed below
- Parameter-Efficient Sparsity Crafting From Dense to Mixture-of-Experts for Instruction Tuning on General Tasks☆142Updated 6 months ago
- Repo for Rho-1: Token-level Data Selection & Selective Pretraining of LLMs.☆406Updated 11 months ago
- ☆253Updated last year
- Implementation of 🥥 Coconut, Chain of Continuous Thought, in Pytorch☆162Updated 3 months ago
- Code for "Critique Fine-Tuning: Learning to Critique is More Effective than Learning to Imitate"☆131Updated last month
- This is the official repository for Inheritune.☆111Updated last month
- ☆172Updated last year
- Benchmarking LLMs with Challenging Tasks from Real Users☆220Updated 5 months ago
- ☆89Updated 3 weeks ago
- ☆182Updated this week
- Code and Data for "Long-context LLMs Struggle with Long In-context Learning" [TMLR2025]☆104Updated last month
- Unofficial implementation for the paper "Mixture-of-Depths: Dynamically allocating compute in transformer-based language models"☆156Updated 9 months ago
- ☆125Updated last year
- Automated Identification of Redundant Layer Blocks for Pruning in Large Language Models☆228Updated 11 months ago
- Code for In-context Vectors: Making In Context Learning More Effective and Controllable Through Latent Space Steering☆167Updated last month
- [NeurIPS 2024] Official Repository of The Mamba in the Llama: Distilling and Accelerating Hybrid Models☆209Updated last month
- Official repository for "Scaling Retrieval-Based Langauge Models with a Trillion-Token Datastore".☆195Updated last week
- Implementation of CALM from the paper "LLM Augmented LLMs: Expanding Capabilities through Composition", out of Google Deepmind☆174Updated 6 months ago
- ☆163Updated last month
- Q-GaLore: Quantized GaLore with INT4 Projection and Layer-Adaptive Low-Rank Gradients.☆195Updated 8 months ago
- ☆195Updated 4 months ago
- [ACL'24] Selective Reflection-Tuning: Student-Selected Data Recycling for LLM Instruction-Tuning☆352Updated 7 months ago
- Official implementation of "DoRA: Weight-Decomposed Low-Rank Adaptation"☆123Updated 11 months ago
- Advancing Language Model Reasoning through Reinforcement Learning and Inference Scaling☆97Updated 2 months ago
- "Improving Mathematical Reasoning with Process Supervision" by OPENAI☆108Updated last month
- Spherical Merge Pytorch/HF format Language Models with minimal feature loss.☆119Updated last year
- Repo for "LoLCATs: On Low-Rank Linearizing of Large Language Models"☆229Updated 2 months ago
- [ICLR 2024 Spotlight] Code for the paper "Merge, Then Compress: Demystify Efficient SMoE with Hints from Its Routing Policy"☆77Updated 10 months ago
- Implementation of the LongRoPE: Extending LLM Context Window Beyond 2 Million Tokens Paper☆129Updated 8 months ago
- State-of-the-art Parameter-Efficient MoE Fine-tuning Method☆153Updated 7 months ago