gpustack / llama-boxLinks
LM inference server implementation based on *.cpp.
☆279Updated last month
Alternatives and similar repositories for llama-box
Users that are interested in llama-box are comparing it to the libraries listed below
Sorting:
- Review/Check GGUF files and estimate the memory usage and maximum tokens per second.☆208Updated last month
- A text-to-speech and speech-to-text server compatible with the OpenAI API, supporting Whisper, FunASR, Bark, and CosyVoice backends.☆163Updated 2 months ago
- run DeepSeek-R1 GGUFs on KTransformers☆252Updated 7 months ago
- xllamacpp - a Python wrapper of llama.cpp☆60Updated last week
- Library for model distillation☆153Updated last month
- ☆93Updated 3 months ago
- automatically quant GGUF models☆204Updated last week
- Pure C++ implementation of several models for real-time chatting on your computer (CPU & GPU)☆716Updated last week
- ☆105Updated 2 weeks ago
- Port of Facebook's LLaMA model in C/C++☆59Updated 5 months ago
- A proxy server for multiple ollama instances with Key security☆499Updated 2 weeks ago
- The main repository for building Pascal-compatible versions of ML applications and libraries.☆132Updated last month
- llmbasedos — Local-First OS Where Your AI Agents Wake Up and Work☆279Updated last month
- An OpenAI API compatible API for chat with image input and questions about the images. aka Multimodal.☆259Updated 7 months ago
- LLM inference in C/C++☆102Updated last month
- Open Source Text Embedding Models with OpenAI Compatible API☆160Updated last year
- Cook up amazing multimodal AI applications effortlessly with MiniCPM-o☆202Updated last week
- The LLM API Benchmark Tool is a flexible Go-based utility designed to measure and analyze the performance of OpenAI-compatible API endpoi…☆43Updated 3 weeks ago
- A memory framework for Large Language Models and Agents.☆183Updated 9 months ago
- A command-line interface tool for serving LLM using vLLM.☆425Updated last month
- The Fastest Way to Fine-Tune LLMs Locally☆321Updated 6 months ago
- ☆207Updated last month
- InferX: Inference as a Service Platform☆136Updated this week
- Download models from the Ollama library, without Ollama☆100Updated 10 months ago
- ☆49Updated last week
- CPU inference for the DeepSeek family of large language models in C++☆313Updated last week
- A real-time speech-to-speech chatbot powered by Whisper Small, Llama 3.2, and Kokoro-82M.☆240Updated 8 months ago
- Self-hosted huggingface mirror service. 自建huggingface镜像服务。☆195Updated 2 months ago
- A development-oriented visualization toolkit☆225Updated last week
- The latest graphrag interface is used, using the local ollama to provide the LLM interface.Support for using the pip installation☆156Updated last year