gpustack / gguf-parser-go
Review/Check GGUF files and estimate the memory usage and maximum tokens per second.
☆47Updated this week
Related projects ⓘ
Alternatives and complementary repositories for gguf-parser-go
- HTTP proxy for on-demand model loading with llama.cpp (or other OpenAI compatible backends)☆41Updated this week
- automatically quant GGUF models☆140Updated this week
- Serving LLMs in the HF-Transformers format via a PyFlask API☆68Updated 2 months ago
- ☆128Updated this week
- Dagger functions to import Hugging Face GGUF models into a local ollama instance and optionally push them to ollama.com.☆110Updated 5 months ago
- This small API downloads and exposes access to NeuML's txtai-wikipedia and full wikipedia datasets, taking in a query and returning full …☆51Updated this week
- A python application that routes incoming prompts to an LLM by category, and can support a single incoming connection from a front end to…☆167Updated this week
- Gradio based tool to run opensource LLM models directly from Huggingface☆87Updated 4 months ago
- Comparison of the output quality of quantization methods, using Llama 3, transformers, GGUF, EXL2.☆126Updated 6 months ago
- Something similar to Apple Intelligence?☆57Updated 4 months ago
- ☆65Updated last month
- ☆149Updated 4 months ago
- A fast batching API to serve LLM models☆172Updated 6 months ago
- Dataset Crafting w/ RAG/Wikipedia ground truth and Efficient Fine-Tuning Using MLX and Unsloth. Includes configurable dataset annotation …☆162Updated 4 months ago
- ☆112Updated this week
- Practical and advanced guide to LLMOps. It provides a solid understanding of large language models’ general concepts, deployment techniqu…☆52Updated 3 months ago
- An extension that lets the AI take the wheel, allowing it to use the mouse and keyboard, recognize UI elements, and prompt itself :3...no…☆96Updated 3 weeks ago
- Easily view and modify JSON datasets for large language models☆62Updated last month
- idea: https://github.com/nyxkrage/ebook-groupchat/☆82Updated 3 months ago
- A python package for developing AI applications with local LLMs.☆140Updated 4 months ago
- CaSIL is an advanced natural language processing system that implements a sophisticated four-layer semantic analysis architecture. It pro…☆62Updated 2 weeks ago
- klmbr - a prompt pre-processing technique to break through the barrier of entropy while generating text with LLMs☆58Updated last month
- GPU Power and Performance Manager☆48Updated last month
- A simple light terminal style chat app that lets you use connect to your local llama.cpp server☆27Updated 4 months ago
- Experimental LLM Inference UX to aid in creative writing☆106Updated 4 months ago
- Parse files (e.g. code repos) and websites to clipboard or a file for ingestions by AI / LLMs☆63Updated this week
- Lightweight, standalone, multi-platform, and privacy focused local LLM chat interface with optional encryption☆55Updated this week
- ☆39Updated 9 months ago
- Guide on text completion large language model fine-tuning, including example scripts and training data acquiring.☆44Updated 6 months ago
- A simple experiment on letting two local LLM have a conversation about anything!☆91Updated 4 months ago