jorahn / llama-int8Links
Quantized inference code for LLaMA models
☆13Updated 2 years ago
Alternatives and similar repositories for llama-int8
Users that are interested in llama-int8 are comparing it to the libraries listed below
Sorting:
- A library for incremental loading of large PyTorch checkpoints☆56Updated 2 years ago
- Trying to deconstruct RWKV in understandable terms☆14Updated 2 years ago
- Preprint: Sheared LLaMA: Accelerating Language Model Pre-training via Structured Pruning☆28Updated 2 years ago
- A more memory-efficient rewrite of the HF transformers implementation of Llama for use with quantized weights.☆64Updated 2 years ago
- Embeddings focused small version of Llama NLP model☆107Updated 2 years ago
- Simple LLM inference server☆20Updated last year
- ☆27Updated 2 years ago
- Steer LLM outputs towards a certain topic/subject and enhance response capabilities using activation engineering by adding steering vecto…☆44Updated last year
- Pressure testing the context window of open LLMs☆25Updated last year
- Stable diffusion google colab kernel☆10Updated 3 years ago
- The first AI artist☆32Updated 2 years ago
- Experimental sampler to make LLMs more creative☆31Updated 2 years ago
- Landmark Attention: Random-Access Infinite Context Length for Transformers QLoRA☆124Updated 2 years ago
- ☆52Updated 2 years ago
- Command-line script for inferencing from models such as falcon-7b-instruct☆75Updated 2 years ago
- Chatbot that answers frequently asked questions in French, English, and Tunisian using the Rasa NLU framework and RWKV-4-Raven☆13Updated 2 years ago
- ☆40Updated 2 years ago
- The GeoV model is a large langauge model designed by Georges Harik and uses Rotary Positional Embeddings with Relative distances (RoPER).…☆121Updated 2 years ago
- Framework agnostic python runtime for RWKV models☆147Updated 2 years ago
- Demonstration that finetuning RoPE model on larger sequences than the pre-trained model adapts the model context limit☆63Updated 2 years ago
- Image Generation API Server - Similar to https://text-generator.io but for images☆51Updated 5 months ago
- A playground to make it easy to try crazy things☆33Updated 2 months ago
- ☆20Updated 4 years ago
- LLM Divergent Thinking Creativity Benchmark. LLMs generate 25 unique words that start with a given letter with no connections to each oth…☆35Updated 10 months ago
- Drop in replacement for OpenAI, but with Open models.☆154Updated 2 years ago
- Use Datasette to explore LAION improved_aesthetics_6plus training data used by Stable DIffusion☆59Updated 2 years ago
- 🤗Transformers: State-of-the-art Natural Language Processing for Pytorch and TensorFlow 2.0.☆56Updated 4 years ago
- Full finetuning of large language models without large memory requirements☆94Updated 4 months ago
- GRDN.AI app for garden optimization☆69Updated 2 months ago
- inference code for mixtral-8x7b-32kseqlen☆105Updated 2 years ago