Code for PHATGOOSE introduced in "Learning to Route Among Specialized Experts for Zero-Shot Generalization"
☆91Feb 27, 2024Updated 2 years ago
Alternatives and similar repositories for phatgoose
Users that are interested in phatgoose are comparing it to the libraries listed below
Sorting:
- ☆30Sep 28, 2023Updated 2 years ago
- My Gen AI research☆11Jun 3, 2024Updated last year
- A library for squeakily cleaning and filtering language datasets.☆50Jul 10, 2023Updated 2 years ago
- Code for NOLA, an implementation of "nola: Compressing LoRA using Linear Combination of Random Basis"☆57Aug 25, 2024Updated last year
- Low-Rank adapter extraction for fine-tuned transformers models☆180May 2, 2024Updated last year
- ☆415Nov 2, 2023Updated 2 years ago
- Building modular LMs with parameter-efficient fine-tuning.☆114Jan 18, 2026Updated last month
- The official repo for "LLoCo: Learning Long Contexts Offline"☆118Jun 15, 2024Updated last year
- Load multiple LoRA modules simultaneously and automatically switch the appropriate combination of LoRA modules to generate the best answe…☆159Feb 9, 2024Updated 2 years ago
- ☆37Nov 27, 2025Updated 3 months ago
- QLoRA with Enhanced Multi GPU Support☆38Aug 8, 2023Updated 2 years ago
- a pipeline for using api calls to agnostically convert unstructured data into structured training data☆32Sep 22, 2024Updated last year
- ☆68May 26, 2024Updated last year
- Code and data for "StructLM: Towards Building Generalist Models for Structured Knowledge Grounding" (COLM 2024)☆76Oct 19, 2024Updated last year
- A repository for research on medium sized language models.☆78May 23, 2024Updated last year
- A library for simplifying training with multi gpu setups in the HuggingFace / PyTorch ecosystem.☆16Jan 9, 2026Updated last month
- ☆13May 9, 2024Updated last year
- Mixture of Expert (MoE) techniques for enhancing LLM performance through expert-driven prompt mapping and adapter combinations.☆12Feb 11, 2024Updated 2 years ago
- A chat implementation for FastHTML☆11Sep 14, 2025Updated 5 months ago
- ☆274Oct 31, 2023Updated 2 years ago
- Spherical Merge Pytorch/HF format Language Models with minimal feature loss.☆145Sep 10, 2023Updated 2 years ago
- CUDA and Triton implementations of Flash Attention with SoftmaxN.☆73May 26, 2024Updated last year
- Codebase for Merging Language Models (ICML 2024)☆863May 5, 2024Updated last year
- Comprehensive analysis of difference in performance of QLora, Lora, and Full Finetunes.☆83Sep 10, 2023Updated 2 years ago
- ☆10Oct 24, 2024Updated last year
- ☆18Apr 3, 2023Updated 2 years ago
- ☆12Feb 11, 2026Updated 3 weeks ago
- MPI Code Generation through Domain-Specific Language Models☆14Nov 19, 2024Updated last year
- ☆10Apr 16, 2021Updated 4 years ago
- YourAICHAT☆13Aug 16, 2023Updated 2 years ago
- Parameter-Efficient Sparsity Crafting From Dense to Mixture-of-Experts for Instruction Tuning on General Tasks (EMNLP'24)☆145Sep 20, 2024Updated last year
- Plug n Play GBNF Compiler for llama.cpp☆28Nov 8, 2023Updated 2 years ago
- ReBase: Training Task Experts through Retrieval Based Distillation☆29Feb 5, 2025Updated last year
- Evaluating LLMs with fewer examples☆169Apr 12, 2024Updated last year
- ☆91Aug 18, 2024Updated last year
- Parameter-Efficient Sparsity Crafting From Dense to Mixture-of-Experts for Instruction Tuning on General Tasks☆31May 22, 2024Updated last year
- Efficient Scaling laws and collaborative pretraining.☆21Sep 18, 2025Updated 5 months ago
- Fine-Tuning Pre-trained Transformers into Decaying Fast Weights☆19Oct 9, 2022Updated 3 years ago
- ModuleFormer is a MoE-based architecture that includes two different types of experts: stick-breaking attention heads and feedforward exp…☆226Sep 18, 2025Updated 5 months ago