cognitivecomputations / extract-expert
Extract a single expert from a Mixture Of Experts model using slerp interpolation.
☆17Updated 5 months ago
Related projects ⓘ
Alternatives and complementary repositories for extract-expert
- ☆64Updated 5 months ago
- ☆104Updated 8 months ago
- entropix style sampling + GUI☆25Updated 3 weeks ago
- ☆20Updated last year
- ☆72Updated last year
- Just a bunch of benchmark logs for different LLMs☆115Updated 3 months ago
- ☆28Updated this week
- Let's create synthetic textbooks together :)☆70Updated 9 months ago
- autologic is a Python package that implements the SELF-DISCOVER framework proposed in the paper SELF-DISCOVER: Large Language Models Self…☆57Updated 9 months ago
- ☆150Updated 4 months ago
- Official homepage for "Self-Harmonized Chain of Thought"☆83Updated 2 months ago
- 5X faster 60% less memory QLoRA finetuning☆21Updated 5 months ago
- A python package for serving LLM on OpenAI-compatible API endpoints with prompt caching using MLX.☆55Updated last week
- An easy-to-understand framework for LLM samplers that rewind and revise generated tokens☆113Updated 3 weeks ago
- An unsupervised model merging algorithm for Transformers-based language models.☆100Updated 6 months ago
- ☆48Updated last year
- This is our own implementation of 'Layer Selective Rank Reduction'☆232Updated 5 months ago
- Scripts to create your own moe models using mlx☆86Updated 8 months ago
- look how they massacred my boy☆58Updated last month
- ReDel is a toolkit for researchers and developers to build, iterate on, and analyze recursive multi-agent systems. (EMNLP 2024 Demo)☆64Updated this week
- Parameter-Efficient Sparsity Crafting From Dense to Mixture-of-Experts for Instruction Tuning on General Tasks☆31Updated 6 months ago
- ☆53Updated 5 months ago
- Fast approximate inference on a single GPU with sparsity aware offloading☆38Updated 10 months ago
- Easily view and modify JSON datasets for large language models☆62Updated last month
- ☆41Updated 2 weeks ago
- ☆93Updated last month
- A framework for evaluating function calls made by LLMs☆35Updated 3 months ago
- LLM-Training-API: Including Embeddings & ReRankers, mergekit, LaserRMT☆26Updated 9 months ago
- Low-Rank adapter extraction for fine-tuned transformers model☆162Updated 6 months ago
- ☆37Updated 11 months ago