Alternatives and similar repositories for llama-box

Users that are interested in llama-box are comparing it to the libraries listed below

• gpustack / gguf-parser-go
  Review/Check GGUF files and estimate the memory usage and maximum tokens per second.
  ☆177Updated last week
• gpustack / vox-box
  A text-to-speech and speech-to-text server compatible with the OpenAI API, supporting Whisper, FunASR, Bark, and CosyVoice backends.
  ☆124Updated 2 weeks ago
• ubergarm / r1-ktransformers-guide
  run DeepSeek-R1 GGUFs on KTransformers
  ☆236Updated 3 months ago
• xorbitsai / xllamacpp
  xllamacpp - a Python wrapper of llama.cpp
  ☆44Updated last week
• horus-ai-labs / DistillFlow
  Library for model distillation
  ☆144Updated 4 months ago
• shell-nlp / gpt_server
  gpt_server是一个用于生产级部署LLMs、Embedding、Reranker、ASR和TTS的开源框架。
  ☆194Updated this week
• and270 / thinking_effort_processor
  ☆90Updated 3 months ago
• KylinMountain / markify
  Convert files into markdown to help RAG or LLM understand, based on markitdown and MinerU, which could provide high quality pdf parser.
  ☆108Updated 2 months ago
• akx / ollama-dl
  Download models from the Ollama library, without Ollama
  ☆86Updated 7 months ago
• modelscope / dash-infer
  DashInfer is a native LLM inference engine aiming to deliver industry-leading performance atop various hardware architectures, including …
  ☆256Updated 3 weeks ago
• 1694439208 / GOT-OCR-Inference
  研究GOT-OCR-项目落地加速，不限语言
  ☆60Updated 8 months ago
• Yoosu-L / llmapibenchmark
  The LLM API Benchmark Tool is a flexible Go-based utility designed to measure and analyze the performance of OpenAI-compatible API endpoi…
  ☆27Updated 4 months ago
• andrewkchan / deepseek.cpp
  CPU inference for the DeepSeek family of large language models in C++
  ☆302Updated 3 weeks ago
• tc-mb / llama.cpp
  Port of Facebook's LLaMA model in C/C++
  ☆96Updated 2 weeks ago
• wade1010 / graphrag-ui
  The latest graphrag interface is used, using the local ollama to provide the LLM interface.Support for using the pip installation
  ☆153Updated 8 months ago
• limcheekin / open-text-embeddings
  Open Source Text Embedding Models with OpenAI Compatible API
  ☆153Updated 11 months ago
• foldl / chatllm.cpp
  Pure C++ implementation of several models for real-time chatting on your computer (CPU & GPU)
  ☆634Updated this week
• HimariO / llama.cpp.qwen2.5vl
  Port of Facebook's LLaMA model in C/C++
  ☆52Updated last month
• thad0ctor / llama-server-launcher
  ☆96Updated last week
• lework / llm-benchmark
  LLM 并发性能测试工具，支持自动化压力测试和性能报告生成。
  ☆93Updated 3 months ago
• leafspark / AutoGGUF
  automatically quant GGUF models
  ☆184Updated last week
• 01-ai / Descartes
  ☆109Updated last year
• sugarforever / peanut-shell
  ☆53Updated 6 months ago
• QwenLM / qwen.cpp
  C++ implementation of Qwen-LM
  ☆595Updated 6 months ago
• chu-tianxiang / vllm-gptq
  A high-throughput and memory-efficient inference and serving engine for LLMs
  ☆131Updated last year
• ModelCloud / GPTQModel
  Production ready LLM model compression/quantization toolkit with hw accelerated inference support for both cpu/gpu via HF, vLLM, and SGLa…
  ☆633Updated this week
• OpenBMB / CPM.cu
  CPM.cu is a lightweight, high-performance CUDA implementation for LLMs, optimized for end-device inference and featuring cutting-edge tec…
  ☆151Updated this week
• AaronFeng753 / Better-Qwen3
  Auto Thinking Mode switch for Qwen3 in Open webui
  ☆65Updated last month
• ikawrakow / ik_llama.cpp
  llama.cpp fork with additional SOTA quants and improved performance
  ☆608Updated this week
• IEIT-Yuan / Yuan2.0-M32
  Mixture-of-Experts (MoE) Language Model
  ☆189Updated 9 months ago