turboderp / exllama
A more memory-efficient rewrite of the HF transformers implementation of Llama for use with quantized weights.
☆2,760Updated last year
Related projects ⓘ
Alternatives and complementary repositories for exllama
- A fast inference library for running LLMs locally on modern consumer-class GPUs☆3,680Updated this week
- 4 bits quantization of LLaMA using GPTQ☆2,998Updated 4 months ago
- An easy-to-use LLMs quantization package with user-friendly apis, based on GPTQ algorithm.☆4,497Updated last month
- Python bindings for the Transformer models implemented in C/C++ using GGML library.☆1,814Updated 9 months ago
- Code for the ICLR 2023 paper "GPTQ: Accurate Post-training Quantization of Generative Pretrained Transformers".☆1,941Updated 7 months ago
- Large Language Model Text Generation Inference☆9,122Updated this week
- Large language models (LLMs) made easy, EasyLM is a one stop solution for pre-training, finetuning, evaluating and serving LLMs in JAX/Fl…☆2,409Updated 3 months ago
- Accessible large language models via k-bit quantization for PyTorch.☆6,299Updated this week
- AutoAWQ implements the AWQ algorithm for 4-bit quantization with a 2x speedup during inference. Documentation:☆1,765Updated this week
- Alpaca dataset from Stanford, cleaned and curated☆1,519Updated last year
- Customizable implementation of the self-instruct paper.☆1,024Updated 8 months ago
- Tools for merging pretrained large language models.☆4,816Updated 2 weeks ago
- Simple UI for LLM Model Finetuning☆2,045Updated 10 months ago
- MII makes low-latency and high-throughput inference possible, powered by DeepSpeed.☆1,904Updated this week
- Go ahead and axolotl questions☆7,930Updated this week
- SGLang is a fast serving framework for large language models and vision language models.☆6,127Updated this week
- Python bindings for llama.cpp☆8,141Updated this week
- The RedPajama-Data repository contains code for preparing large datasets for training large language models.☆4,571Updated last month
- INT4/INT5/INT8 and FP16 inference on CPU for RWKV language model☆1,424Updated 3 months ago
- [MLSys 2024 Best Paper Award] AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration☆2,526Updated last month
- ☆1,430Updated last year
- A collection of modular datasets generated by GPT-4, General-Instruct - Roleplay-Instruct - Code-Instruct - and Toolformer☆1,618Updated last year
- LLMs build upon Evol Insturct: WizardLM, WizardCoder, WizardMath☆9,269Updated 3 months ago
- LLaMA: Open and Efficient Foundation Language Models☆2,807Updated last year
- Large-scale LLM inference engine☆1,134Updated this week
- LLM as a Chatbot Service☆3,296Updated last year
- QLoRA: Efficient Finetuning of Quantized LLMs☆10,059Updated 5 months ago
- S-LoRA: Serving Thousands of Concurrent LoRA Adapters☆1,755Updated 10 months ago
- Multi-LoRA inference server that scales to 1000s of fine-tuned LLMs☆2,205Updated this week