matt-c1 / llama-3-quant-comparison
Comparison of the output quality of quantization methods, using Llama 3, transformers, GGUF, EXL2.
☆126Updated 6 months ago
Related projects ⓘ
Alternatives and complementary repositories for llama-3-quant-comparison
- A fast batching API to serve LLM models☆172Updated 6 months ago
- Low-Rank adapter extraction for fine-tuned transformers model☆162Updated 6 months ago
- ☆227Updated last month
- Automated Identification of Redundant Layer Blocks for Pruning in Large Language Models☆196Updated 6 months ago
- This is our own implementation of 'Layer Selective Rank Reduction'☆232Updated 5 months ago
- ☆149Updated 4 months ago
- ☆65Updated last month
- A multimodal, function calling powered LLM webui.☆208Updated last month
- idea: https://github.com/nyxkrage/ebook-groupchat/☆82Updated 3 months ago
- A python application that routes incoming prompts to an LLM by category, and can support a single incoming connection from a front end to…☆167Updated this week
- Dataset Crafting w/ RAG/Wikipedia ground truth and Efficient Fine-Tuning Using MLX and Unsloth. Includes configurable dataset annotation …☆162Updated 4 months ago
- Simple Python library/structure to ablate features in LLMs which are supported by TransformerLens☆333Updated 5 months ago
- Q-GaLore: Quantized GaLore with INT4 Projection and Layer-Adaptive Low-Rank Gradients.☆173Updated 4 months ago
- Generate Synthetic Data Using OpenAI, MistralAI or AnthropicAI☆221Updated 6 months ago
- 1.58-bit LLaMa model☆79Updated 7 months ago
- Fast parallel LLM inference for MLX☆149Updated 4 months ago
- Web UI for ExLlamaV2☆445Updated last month
- An unsupervised model merging algorithm for Transformers-based language models.☆100Updated 6 months ago
- Experimental LLM Inference UX to aid in creative writing☆106Updated 4 months ago
- ☆118Updated 3 months ago
- Easily view and modify JSON datasets for large language models☆62Updated last month
- A pipeline parallel training script for LLMs.☆83Updated this week
- 🚀 Retrieval Augmented Generation (RAG) with txtai. Combine search and LLMs to find insights with your own data.☆281Updated this week
- An Open Source Toolkit For LLM Distillation☆356Updated 2 months ago
- Let's create synthetic textbooks together :)☆70Updated 9 months ago
- HTTP proxy for on-demand model loading with llama.cpp (or other OpenAI compatible backends)☆41Updated this week
- ☆128Updated this week
- ☆104Updated 8 months ago
- AI management tool☆107Updated last week
- klmbr - a prompt pre-processing technique to break through the barrier of entropy while generating text with LLMs☆58Updated last month