Alternatives and similar repositories for llama-box

Users that are interested in llama-box are comparing it to the libraries listed below

• gpustack / gguf-parser-go
  Review/Check GGUF files and estimate the memory usage and maximum tokens per second.
  ☆200Updated last week
• gpustack / vox-box
  A text-to-speech and speech-to-text server compatible with the OpenAI API, supporting Whisper, FunASR, Bark, and CosyVoice backends.
  ☆154Updated last month
• ubergarm / r1-ktransformers-guide
  run DeepSeek-R1 GGUFs on KTransformers
  ☆250Updated 5 months ago
• xorbitsai / xllamacpp
  xllamacpp - a Python wrapper of llama.cpp
  ☆52Updated this week
• horus-ai-labs / DistillFlow
  Library for model distillation
  ☆150Updated 6 months ago
• foldl / chatllm.cpp
  Pure C++ implementation of several models for real-time chatting on your computer (CPU & GPU)
  ☆680Updated last week
• and270 / thinking_effort_processor
  ☆91Updated last month
• thad0ctor / llama-server-launcher
  ☆104Updated 3 weeks ago
• leafspark / AutoGGUF
  automatically quant GGUF models
  ☆196Updated this week
• andrewkchan / deepseek.cpp
  CPU inference for the DeepSeek family of large language models in C++
  ☆310Updated 2 months ago
• ParisNeo / ollama_proxy_server
  A proxy server for multiple ollama instances with Key security
  ☆483Updated 3 weeks ago
• matatonic / openedai-vision
  An OpenAI API compatible API for chat with image input and questions about the images. aka Multimodal.
  ☆259Updated 5 months ago
• iluxu / llmbasedos
  llmbasedos — Local-First OS Where Your AI Agents Wake Up and Work
  ☆271Updated last week
• ikawrakow / ik_llama.cpp
  llama.cpp fork with additional SOTA quants and improved performance
  ☆1,096Updated last week
• rag-wtf / open-text-embeddings
  Open Source Text Embedding Models with OpenAI Compatible API
  ☆159Updated last year
• unslothai / llama.cpp
  LLM inference in C/C++
  ☆101Updated this week
• HimariO / llama.cpp.qwen2.5vl
  Port of Facebook's LLaMA model in C/C++
  ☆56Updated 4 months ago
• Yoosu-L / llmapibenchmark
  The LLM API Benchmark Tool is a flexible Go-based utility designed to measure and analyze the performance of OpenAI-compatible API endpoi…
  ☆38Updated last week
• yoziru / nextjs-vllm-ui
  Fully-featured, beautiful web interface for vLLM - built with NextJS.
  ☆150Updated 3 months ago
• inferx-net / inferx
  InferX is a Inference Function as a Service Platform
  ☆129Updated last week
• sasha0552 / pascal-pkgs-ci
  The main repository for building Pascal-compatible versions of ML applications and libraries.
  ☆121Updated this week
• chu-tianxiang / vllm-gptq
  A high-throughput and memory-efficient inference and serving engine for LLMs
  ☆132Updated last year
• fairydreaming / llama.cpp
  LLM inference in C/C++
  ☆21Updated 5 months ago
• SearchSavior / OpenArc
  Lightweight Inference server for OpenVINO
  ☆202Updated this week
• MaxHastings / Kolo
  The Fastest Way to Fine-Tune LLMs Locally
  ☆316Updated 5 months ago
• vtuber-plan / olah
  Self-hosted huggingface mirror service. 自建huggingface镜像服务。
  ☆189Updated last month
• phildougherty / qwen2.5-VL-inference-openai
  Inference service for Qwen2.5-VL-7b model
  ☆194Updated 5 months ago
• matteoserva / GraphLLM
  ☆209Updated last month
• tcsenpai / multi1
  multi1: create o1-like reasoning chains with multiple AI providers (and locally). Supports LiteLLM as backend too for 100+ providers at o…
  ☆350Updated 7 months ago
• NimbleEdge / sparse_transformers
  Sparse Inferencing for transformer based LLMs
  ☆197Updated 2 weeks ago