turboderp / exllama
A more memory-efficient rewrite of the HF transformers implementation of Llama for use with quantized weights.
☆2,759Updated last year
Related projects ⓘ
Alternatives and complementary repositories for exllama
- A fast inference library for running LLMs locally on modern consumer-class GPUs☆3,629Updated last week
- 4 bits quantization of LLaMA using GPTQ☆2,993Updated 3 months ago
- An easy-to-use LLMs quantization package with user-friendly apis, based on GPTQ algorithm.☆4,458Updated last month
- Python bindings for the Transformer models implemented in C/C++ using GGML library.☆1,811Updated 9 months ago
- Code for the ICLR 2023 paper "GPTQ: Accurate Post-training Quantization of Generative Pretrained Transformers".☆1,922Updated 7 months ago
- Accessible large language models via k-bit quantization for PyTorch.☆6,244Updated this week
- AutoAWQ implements the AWQ algorithm for 4-bit quantization with a 2x speedup during inference. Documentation:☆1,743Updated last month
- Simple UI for LLM Model Finetuning☆2,046Updated 10 months ago
- Large language models (LLMs) made easy, EasyLM is a one stop solution for pre-training, finetuning, evaluating and serving LLMs in JAX/Fl…☆2,401Updated 2 months ago
- The RedPajama-Data repository contains code for preparing large datasets for training large language models.☆4,564Updated 3 weeks ago
- Alpaca dataset from Stanford, cleaned and curated☆1,515Updated last year
- INT4/INT5/INT8 and FP16 inference on CPU for RWKV language model☆1,420Updated 3 months ago
- Customizable implementation of the self-instruct paper.☆1,018Updated 8 months ago
- LLaMA: Open and Efficient Foundation Language Models☆2,807Updated last year
- Python bindings for llama.cpp☆8,072Updated this week
- Tools for merging pretrained large language models.☆4,788Updated this week
- OpenLLaMA, a permissively licensed open source reproduction of Meta AI’s LLaMA 7B trained on the RedPajama dataset☆7,383Updated last year
- Build, customize and control you own LLMs. From data pre-processing to fine-tuning, xTuring provides an easy way to personalize open-sour…☆2,610Updated last month
- Large Language Model Text Generation Inference☆9,011Updated this week
- LLM as a Chatbot Service☆3,290Updated 11 months ago
- S-LoRA: Serving Thousands of Concurrent LoRA Adapters☆1,745Updated 9 months ago
- Go ahead and axolotl questions☆7,858Updated this week
- Aligning pretrained language models with instruction data generated by themselves.☆4,133Updated last year
- [ACL 2023] One Embedder, Any Task: Instruction-Finetuned Text Embeddings☆1,864Updated 2 months ago
- Fast inference engine for Transformer models☆3,383Updated this week
- A collection of modular datasets generated by GPT-4, General-Instruct - Roleplay-Instruct - Code-Instruct - and Toolformer☆1,618Updated last year
- Multi-LoRA inference server that scales to 1000s of fine-tuned LLMs☆2,178Updated this week
- [MLSys 2024 Best Paper Award] AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration☆2,498Updated 3 weeks ago
- Large-scale LLM inference engine☆1,101Updated this week