Alternatives and similar repositories for llama-box

Users that are interested in llama-box are comparing it to the libraries listed below

• gpustack / gguf-parser-go
  Review/Check GGUF files and estimate the memory usage and maximum tokens per second.
  ☆232Updated 2 weeks ago
• gpustack / vox-box
  A text-to-speech and speech-to-text server compatible with the OpenAI API, supporting Whisper, FunASR, Bark, and CosyVoice backends.
  ☆190Updated 3 weeks ago
• ubergarm / r1-ktransformers-guide
  run DeepSeek-R1 GGUFs on KTransformers
  ☆259Updated 10 months ago
• horus-ai-labs / DistillFlow
  Library for model distillation
  ☆160Updated 4 months ago
• xorbitsai / xllamacpp
  xllamacpp - a Python wrapper of llama.cpp
  ☆70Updated last week
• foldl / chatllm.cpp
  Pure C++ implementation of several models for real-time chatting on your computer (CPU & GPU)
  ☆770Updated last week
• and270 / thinking_effort_processor
  ☆94Updated 6 months ago
• andrewkchan / deepseek.cpp
  CPU inference for the DeepSeek family of large language models in C++
  ☆317Updated 3 months ago
• sasha0552 / pascal-pkgs-ci
  The main repository for building Pascal-compatible versions of ML applications and libraries.
  ☆162Updated 4 months ago
• rag-wtf / open-text-embeddings
  Open Source Text Embedding Models with OpenAI Compatible API
  ☆165Updated last year
• leafspark / AutoGGUF
  automatically quant GGUF models
  ☆220Updated 3 weeks ago
• micytao / vllm-playground
  A modern web interface for managing and interacting with vLLM servers (www.github.com/vllm-project/vllm). Supports both GPU and CPU modes…
  ☆331Updated this week
• HimariO / llama.cpp.qwen2.5vl
  Port of Facebook's LLaMA model in C/C++
  ☆67Updated 8 months ago
• OpenSQZ / MiniCPM-V-CookBook
  Cook up amazing multimodal AI applications effortlessly with MiniCPM-o
  ☆237Updated last month
• MaxHastings / Kolo
  The Fastest Way to Fine-Tune LLMs Locally
  ☆333Updated last month
• unslothai / llama.cpp
  LLM inference in C/C++
  ☆104Updated last month
• thad0ctor / llama-server-launcher
  ☆107Updated last month
• wade1010 / graphrag-ui
  The latest graphrag interface is used, using the local ollama to provide the LLM interface.Support for using the pip installation
  ☆156Updated last year
• Yoosu-L / llmapibenchmark
  The LLM API Benchmark Tool is a flexible Go-based utility designed to measure and analyze the performance of OpenAI-compatible API endpoi…
  ☆63Updated 2 months ago
• SearchSavior / OpenArc
  Inference engine for Intel devices. Serve LLMs, VLMs, Whisper, Kokoro-TTS, Embedding and Rerank models over OpenAI endpoints.
  ☆274Updated last week
• aikitoria / open-gpu-kernel-modules
  NVIDIA Linux open GPU with P2P support
  ☆112Updated last month
• ParisNeo / ollama_proxy_server
  A proxy server for multiple ollama instances with Key security
  ☆565Updated 2 months ago
• KylinMountain / markify
  Convert files into markdown to help RAG or LLM understand, based on markitdown and MinerU, which could provide high quality pdf parser.
  ☆131Updated 9 months ago
• iluxu / llmbasedos
  llmbasedos — Local-First OS Where Your AI Agents Wake Up and Work
  ☆280Updated 2 weeks ago
• yoziru / nextjs-vllm-ui
  Fully-featured, beautiful web interface for vLLM - built with NextJS.
  ☆170Updated last month
• chisasaw / redcache-ai
  A memory framework for Large Language Models and Agents.
  ☆181Updated last year
• inferx-net / inferx
  InferX: Inference as a Service Platform
  ☆146Updated this week
• vtuber-plan / olah
  Self-hosted huggingface mirror service. 自建huggingface镜像服务。
  ☆212Updated 6 months ago
• matteoserva / GraphLLM
  ☆210Updated 2 weeks ago
• nlzy / vllm-gfx906
  vLLM for AMD gfx906 GPUs, e.g. Radeon VII / MI50 / MI60
  ☆362Updated 3 weeks ago