jorahn / llama-int8Links
Quantized inference code for LLaMA models
☆13Updated 2 years ago
Alternatives and similar repositories for llama-int8
Users that are interested in llama-int8 are comparing it to the libraries listed below
Sorting:
- Embeddings focused small version of Llama NLP model☆104Updated 2 years ago
- ☆40Updated 2 years ago
- Trying to deconstruct RWKV in understandable terms☆14Updated 2 years ago
- Experimental sampler to make LLMs more creative☆31Updated 2 years ago
- A library for incremental loading of large PyTorch checkpoints☆56Updated 2 years ago
- Preprint: Sheared LLaMA: Accelerating Language Model Pre-training via Structured Pruning☆28Updated last year
- An OpenAI API compatible LLM inference server based on ExLlamaV2.☆25Updated last year
- ☆27Updated 2 years ago
- ☆51Updated last year
- Nearly a thousand bash and python scripts I've written over the years.☆124Updated 7 months ago
- ☆39Updated 2 years ago
- Command-line script for inferencing from models such as falcon-7b-instruct☆75Updated 2 years ago
- Framework agnostic python runtime for RWKV models☆146Updated 2 years ago
- LLM Divergent Thinking Creativity Benchmark. LLMs generate 25 unique words that start with a given letter with no connections to each oth…☆34Updated 6 months ago
- Yet Another LLaMA/ALPACA Discord Bot☆70Updated 2 years ago
- Demonstration that finetuning RoPE model on larger sequences than the pre-trained model adapts the model context limit☆63Updated 2 years ago
- The first AI artist☆32Updated 2 years ago
- Just a simple HowTo for https://github.com/johnsmith0031/alpaca_lora_4bit☆32Updated 2 years ago
- Landmark Attention: Random-Access Infinite Context Length for Transformers QLoRA☆124Updated 2 years ago
- WebGPU LLM inference tuned by hand☆151Updated 2 years ago
- Simple LLM inference server☆20Updated last year
- Drop in replacement for OpenAI, but with Open models.☆153Updated 2 years ago
- Fast inference of Instruct tuned LLaMa on your personal devices.☆22Updated 2 years ago
- BlinkDL's RWKV-v4 running in the browser☆47Updated 2 years ago
- Steer LLM outputs towards a certain topic/subject and enhance response capabilities using activation engineering by adding steering vecto…☆43Updated last year
- The GeoV model is a large langauge model designed by Georges Harik and uses Rotary Positional Embeddings with Relative distances (RoPER).…☆121Updated 2 years ago
- ☆23Updated last year
- ☆19Updated 2 years ago
- Command-line script for inferencing from models such as MPT-7B-Chat☆100Updated 2 years ago
- ☆26Updated 2 years ago