gpustack / gguf-parser-goLinks
Review/Check GGUF files and estimate the memory usage and maximum tokens per second.
☆211Updated 2 months ago
Alternatives and similar repositories for gguf-parser-go
Users that are interested in gguf-parser-go are comparing it to the libraries listed below
Sorting:
- LM inference server implementation based on *.cpp.☆286Updated 2 months ago
- Pure C++ implementation of several models for real-time chatting on your computer (CPU & GPU)☆721Updated last week
- A text-to-speech and speech-to-text server compatible with the OpenAI API, supporting Whisper, FunASR, Bark, and CosyVoice backends.☆166Updated 3 months ago
- run DeepSeek-R1 GGUFs on KTransformers☆253Updated 7 months ago
- A proxy server for multiple ollama instances with Key security☆508Updated last week
- Download models from the Ollama library, without Ollama☆103Updated 11 months ago
- VSCode AI coding assistant powered by self-hosted llama.cpp endpoint.☆183Updated 8 months ago
- The LLM API Benchmark Tool is a flexible Go-based utility designed to measure and analyze the performance of OpenAI-compatible API endpoi…☆49Updated last week
- Comparison of Language Model Inference Engines☆231Updated 10 months ago
- ☆93Updated 3 months ago
- Library for model distillation☆152Updated last month
- ☆104Updated 2 months ago
- ☆105Updated last month
- automatically quant GGUF models☆212Updated last week
- LLM inference in C/C++☆102Updated last month
- Cortex.Tensorrt-LLM is a C++ inference library that can be loaded by any server at runtime. It submodules NVIDIA’s TensorRT-LLM for GPU a…☆42Updated last year
- Self-hosted huggingface mirror service. 自建huggingface镜像服务。☆196Updated 3 months ago
- Evaluate and Enhance Your LLM Deployments for Real-World Inference Needs☆655Updated this week
- LLM model quantization (compression) toolkit with hw acceleration support for Nvidia CUDA, AMD ROCm, Intel XPU and Intel/AMD/Apple CPU vi…☆842Updated this week
- llama.cpp fork with additional SOTA quants and improved performance☆1,266Updated this week
- xllamacpp - a Python wrapper of llama.cpp☆60Updated last week
- The Fastest Way to Fine-Tune LLMs Locally☆322Updated 7 months ago
- Open Source Text Embedding Models with OpenAI Compatible API☆160Updated last year
- 🌟 Yi-Coder is a series of open-source code language models that delivers state-of-the-art coding performance with fewer than 10 billion …☆432Updated last year
- Docs for GGUF quantization (unofficial)☆286Updated 3 months ago
- ☆206Updated last month
- The main repository for building Pascal-compatible versions of ML applications and libraries.☆136Updated 2 months ago
- InferX: Inference as a Service Platform☆136Updated this week
- llmbasedos — Local-First OS Where Your AI Agents Wake Up and Work☆276Updated 2 months ago
- Gemma 2 optimized for your local machine.☆376Updated last year