xhedit / quantkitLinks
cli tool to quantize gguf, gptq, awq, hqq and exl2 models
☆76Updated 9 months ago
Alternatives and similar repositories for quantkit
Users that are interested in quantkit are comparing it to the libraries listed below
Sorting:
- ☆116Updated 9 months ago
- Easily view and modify JSON datasets for large language models☆83Updated 4 months ago
- A pipeline parallel training script for LLMs.☆158Updated 5 months ago
- Transplants vocabulary between language models, enabling the creation of draft models for speculative decoding WITHOUT retraining.☆42Updated last month
- Easy to use, High Performant Knowledge Distillation for LLMs☆93Updated 5 months ago
- Gradio based tool to run opensource LLM models directly from Huggingface☆95Updated last year
- Low-Rank adapter extraction for fine-tuned transformers models☆178Updated last year
- A python package for developing AI applications with local LLMs.☆150Updated 9 months ago
- ☆162Updated 2 months ago
- Distributed Inference for mlx LLm☆96Updated last year
- Tcurtsni: Reverse Instruction Chat, ever wonder what your LLM wants to ask you?☆23Updated last year
- ☆67Updated last year
- entropix style sampling + GUI☆27Updated 11 months ago
- automatically quant GGUF models☆204Updated last week
- An unsupervised model merging algorithm for Transformers-based language models.☆106Updated last year
- After my server ui improvements were successfully merged, consider this repo a playground for experimenting, tinkering and hacking around…☆53Updated last year
- ☆24Updated 8 months ago
- ☆102Updated last month
- Forces DeepSeek R1 models to engage in extended reasoning by intercepting early termination tokens.☆19Updated 7 months ago
- run ollama & gguf easily with a single command☆52Updated last year
- ☆51Updated last year
- klmbr - a prompt pre-processing technique to break through the barrier of entropy while generating text with LLMs☆79Updated last year
- An OpenAI API compatible LLM inference server based on ExLlamaV2.☆25Updated last year
- Experimental LLM Inference UX to aid in creative writing☆122Updated 9 months ago
- Create text chunks which end at natural stopping points without using a tokenizer☆26Updated 6 months ago
- ☆132Updated 5 months ago
- LLM based agents with proactive interactions, long-term memory, external tool integration, and local deployment capabilities.☆105Updated 2 months ago
- Glyphs, acting as collaboratively defined symbols linking related concepts, add a layer of multidimensional semantic richness to user-AI …☆51Updated 8 months ago
- GPT-4 Level Conversational QA Trained In a Few Hours☆65Updated last year
- For inferring and serving local LLMs using the MLX framework☆109Updated last year