turboderp-org / exllamav3Links
An optimized quantization and inference library for running LLMs locally on modern consumer-class GPUs
☆512Updated this week
Alternatives and similar repositories for exllamav3
Users that are interested in exllamav3 are comparing it to the libraries listed below
Sorting:
- The official API server for Exllama. OAI compatible, lightweight, and fast.☆1,059Updated this week
- Web UI for ExLlamaV2☆512Updated 7 months ago
- llama.cpp fork with additional SOTA quants and improved performance☆1,220Updated this week
- Comparison of the output quality of quantization methods, using Llama 3, transformers, GGUF, EXL2.☆163Updated last year
- Large-scale LLM inference engine☆1,560Updated this week
- ☆85Updated this week
- LLM Frontend in a single html file☆647Updated 8 months ago
- Easily view and modify JSON datasets for large language models☆83Updated 4 months ago
- automatically quant GGUF models☆203Updated last week
- ☆134Updated 5 months ago
- A daemon that automatically manages the performance states of NVIDIA GPUs.☆97Updated this week
- Model swapping for llama.cpp (or any local OpenAI API compatible server)☆1,615Updated this week
- An OpenAI API compatible API for chat with image input and questions about the images. aka Multimodal.☆260Updated 6 months ago
- Sparse Inferencing for transformer based LLMs☆201Updated last month
- ☆101Updated last month
- A multimodal, function calling powered LLM webui.☆216Updated last year
- A fast batching API to serve LLM models☆187Updated last year
- Croco.Cpp is fork of KoboldCPP infering GGML/GGUF models on CPU/Cuda with KoboldAI's UI. It's powered partly by IK_LLama.cpp, and compati…☆142Updated this week
- ☆316Updated 2 months ago
- Docs for GGUF quantization (unofficial)☆267Updated 2 months ago
- DFloat11: Lossless LLM Compression for Efficient GPU Inference☆542Updated last month
- A pipeline parallel training script for LLMs.☆159Updated 5 months ago
- A local AI companion that uses a collection of free, open source AI models in order to create two virtual companions that will follow you…☆233Updated 2 months ago
- Guaranteed Structured Output from any Language Model via Hierarchical State Machines☆146Updated last week
- Advanced Quantization Algorithm for LLMs and VLMs, with support for CPU, Intel GPU, CUDA and HPU.☆647Updated this week
- Simple Python library/structure to ablate features in LLMs which are supported by TransformerLens☆510Updated last year
- VPTQ, A Flexible and Extreme low-bit quantization algorithm☆659Updated 5 months ago
- ☆51Updated 7 months ago
- AI management tool☆121Updated 10 months ago
- llama.cpp fork with additional SOTA quants and improved performance☆21Updated this week