chrisociepa / allamo
Simple, hackable and fast implementation for training/finetuning medium-sized LLaMA-based models
☆143Updated this week
Related projects: ⓘ
- Falcon LLM ggml framework with CPU and GPU support☆245Updated 7 months ago
- Tune any FALCON in 4-bit☆469Updated last year
- Landmark Attention: Random-Access Infinite Context Length for Transformers QLoRA☆123Updated last year
- Merge Transformers language models by use of gradient parameters.☆193Updated last month
- Tune MPTs☆84Updated last year
- Low-Rank adapter extraction for fine-tuned transformers model☆154Updated 4 months ago
- ☆533Updated 9 months ago
- Load multiple LoRA modules simultaneously and automatically switch the appropriate combination of LoRA modules to generate the best answe…☆139Updated 7 months ago
- 4 bits quantization of LLaMa using GPTQ☆129Updated last year
- QLoRA: Efficient Finetuning of Quantized LLMs☆74Updated 5 months ago
- A bagel, with everything.☆306Updated 5 months ago
- Patch for MPT-7B which allows using and training a LoRA☆58Updated last year
- Instruct-tune Open LLaMA / RedPajama / StableLM models on consumer hardware using QLoRA☆80Updated 9 months ago
- ☆251Updated last year
- Train llama with lora on one 4090 and merge weight of lora to work as stanford alpaca.☆50Updated last year
- ☆453Updated 11 months ago
- Convenient wrapper for fine-tuning and inference of Large Language Models (LLMs) with several quantization techniques (GTPQ, bitsandbytes…☆139Updated 11 months ago
- Spherical Merge Pytorch/HF format Language Models with minimal feature loss.☆107Updated last year
- Reimplementation of the task generation part from the Alpaca paper☆118Updated last year
- ☆409Updated 10 months ago
- Landmark Attention: Random-Access Infinite Context Length for Transformers☆405Updated 8 months ago
- This repository contains code for extending the Stanford Alpaca synthetic instruction tuning to existing instruction-tuned models such as…☆347Updated last year
- This is our own implementation of 'Layer Selective Rank Reduction'☆229Updated 3 months ago
- LLaMa retrieval plugin script using OpenAI's retrieval plugin☆326Updated last year
- Harnessing the Memory Power of the Camelids☆145Updated 11 months ago
- Due to restriction of LLaMA, we try to reimplement BLOOM-LoRA (much less restricted BLOOM license here https://huggingface.co/spaces/bigs…☆183Updated last year
- SoTA Transformers with C-backend for fast inference on your CPU.☆311Updated 9 months ago
- Python bindings for llama.cpp☆199Updated last year
- RWKV infctx trainer, for training arbitary context sizes, to 10k and beyond!☆131Updated last month
- ☆167Updated last year