jorahn / llama-int8
Quantized inference code for LLaMA models
☆13Updated 2 years ago
Alternatives and similar repositories for llama-int8
Users that are interested in llama-int8 are comparing it to the libraries listed below
Sorting:
- Trying to deconstruct RWKV in understandable terms☆14Updated 2 years ago
- A library for incremental loading of large PyTorch checkpoints☆56Updated 2 years ago
- Preprint: Sheared LLaMA: Accelerating Language Model Pre-training via Structured Pruning☆28Updated last year
- Demonstration that finetuning RoPE model on larger sequences than the pre-trained model adapts the model context limit☆63Updated last year
- Steer LLM outputs towards a certain topic/subject and enhance response capabilities using activation engineering by adding steering vecto…☆43Updated last year
- An OpenAI API compatible LLM inference server based on ExLlamaV2.☆25Updated last year
- ☆27Updated last year
- ☆26Updated 2 years ago
- assign color hues to a collection of text fragments based on embeddings☆20Updated 11 months ago
- A utility that downloads your Stable Diffusion images from discord and lets you preview them with Streamlit☆15Updated 2 years ago
- Experimental sampler to make LLMs more creative☆31Updated last year
- Command-line script for inferencing from models such as falcon-7b-instruct☆76Updated last year
- Simple LLM inference server☆20Updated 11 months ago
- Chatbot that answers frequently asked questions in French, English, and Tunisian using the Rasa NLU framework and RWKV-4-Raven☆13Updated last year
- ☆20Updated 3 years ago
- Pressure testing the context window of open LLMs☆24Updated 8 months ago
- Doohickey is a stable diffusion tool for technical artists who want to stay up-to-date with the latest developments in the field.☆39Updated 2 years ago
- ☆19Updated 2 years ago
- Image Generation API Server - Similar to https://text-generator.io but for images☆50Updated 5 months ago
- ☆28Updated last year
- ☆35Updated 2 years ago
- Latent Large Language Models☆18Updated 8 months ago
- Training hybrid models for dummies.☆21Updated 4 months ago
- ☆40Updated 2 years ago
- A more memory-efficient rewrite of the HF transformers implementation of Llama for use with quantized weights.☆64Updated last year
- The first AI artist☆32Updated 2 years ago
- Rust bindings for CTranslate2☆14Updated last year
- ☆15Updated last year
- A playground to make it easy to try crazy things☆33Updated 2 weeks ago
- Accepts a Hugging Face model URL, automatically downloads and quantizes it using Bits and Bytes.☆38Updated last year