gpustack / gguf-parser-goLinks
Review/Check GGUF files and estimate the memory usage and maximum tokens per second.
☆197Updated last week
Alternatives and similar repositories for gguf-parser-go
Users that are interested in gguf-parser-go are comparing it to the libraries listed below
Sorting:
- LM inference server implementation based on *.cpp.☆255Updated last week
- Pure C++ implementation of several models for real-time chatting on your computer (CPU & GPU)☆677Updated this week
- A text-to-speech and speech-to-text server compatible with the OpenAI API, supporting Whisper, FunASR, Bark, and CosyVoice backends.☆150Updated last month
- Download models from the Ollama library, without Ollama☆92Updated 9 months ago
- A proxy server for multiple ollama instances with Key security☆480Updated 3 weeks ago
- Library for model distillation☆150Updated 6 months ago
- VSCode AI coding assistant powered by self-hosted llama.cpp endpoint.☆183Updated 6 months ago
- xllamacpp - a Python wrapper of llama.cpp☆49Updated last week
- Self-hosted huggingface mirror service. 自建huggingface镜像服务。☆186Updated last month
- run DeepSeek-R1 GGUFs on KTransformers☆250Updated 5 months ago
- llmbasedos — Local-First OS Where Your AI Agents Wake Up and Work☆267Updated last week
- The Fastest Way to Fine-Tune LLMs Locally☆316Updated 5 months ago
- llama.cpp fork with additional SOTA quants and improved performance☆1,050Updated this week
- ☆209Updated last month
- automatically quant GGUF models☆195Updated this week
- ☆104Updated 2 weeks ago
- The LLM API Benchmark Tool is a flexible Go-based utility designed to measure and analyze the performance of OpenAI-compatible API endpoi…☆37Updated this week
- Open Source Text Embedding Models with OpenAI Compatible API☆159Updated last year
- InferX is a Inference Function as a Service Platform☆128Updated this week
- LLM inference in C/C++☆101Updated 2 weeks ago
- Fully-featured, beautiful web interface for vLLM - built with NextJS.☆150Updated 3 months ago
- Production ready LLM model compression/quantization toolkit with hw accelerated inference support for both cpu/gpu via HF, vLLM, and SGLa…☆735Updated last week
- Lightweight Inference server for OpenVINO☆198Updated this week
- 🏗️ Fine-tune, build, and deploy open-source LLMs easily!☆467Updated this week
- Code execution utilities for Open WebUI & Ollama☆296Updated 9 months ago
- ☆91Updated last month
- Comparison of Language Model Inference Engines☆228Updated 8 months ago
- Local Qwen3 LLM inference. One easy-to-understand file of C source with no dependencies.☆100Updated last month
- ☆95Updated 7 months ago
- Tool to download models from Huggingface Hub and convert them to GGML/GGUF for llama.cpp☆158Updated 3 months ago