Alternatives and similar repositories for llama-box:

Users that are interested in llama-box are comparing it to the libraries listed below

• gpustack / vox-box
  A text-to-speech and speech-to-text server compatible with the OpenAI API, supporting Whisper, FunASR, Bark, and CosyVoice backends.
  ☆87Updated 2 months ago
• gpustack / gguf-parser-go
  Review/Check GGUF files and estimate the memory usage and maximum tokens per second.
  ☆135Updated 2 weeks ago
• leafspark / AutoGGUF
  automatically quant GGUF models
  ☆164Updated last week
• and270 / thinking_effort_processor
  ☆81Updated 2 weeks ago
• xorbitsai / xllamacpp
  xllamacpp - a Python wrapper of llama.cpp
  ☆29Updated last week
• horus-ai-labs / DistillFlow
  ☆141Updated last month
• ikawrakow / ik_llama.cpp
  llama.cpp fork with additional SOTA quants and improved performance
  ☆222Updated this week
• THUDM / GLM-Edge
  GLM Series Edge Models
  ☆131Updated last month
• modelscope / dash-infer
  DashInfer is a native LLM inference engine aiming to deliver industry-leading performance atop various hardware architectures, including …
  ☆240Updated 3 weeks ago
• IEIT-Yuan / Yuan2.0-M32
  Mixture-of-Experts (MoE) Language Model
  ☆185Updated 6 months ago
• matatonic / openedai-vision
  An OpenAI API compatible API for chat with image input and questions about the images. aka Multimodal.
  ☆243Updated 3 weeks ago
• chu-tianxiang / vllm-gptq
  A high-throughput and memory-efficient inference and serving engine for LLMs
  ☆131Updated 9 months ago
• andrewkchan / deepseek.cpp
  CPU inference for the DeepSeek family of large language models in pure C++
  ☆282Updated last month
• 1694439208 / GOT-OCR-Inference
  研究GOT-OCR-项目落地加速，不限语言
  ☆59Updated 5 months ago
• ubergarm / r1-ktransformers-guide
  run DeepSeek-R1 GGUFs on KTransformers
  ☆212Updated 3 weeks ago
• dataelement / bisheng-unstructured
  bisheng-unstructured library
  ☆42Updated this week
• KylinMountain / markify
  Convert files into markdown to help RAG or LLM understand, based on markitdown and MinerU, which could provide high quality pdf parser.
  ☆74Updated this week
• wade1010 / graphrag-ui
  The latest graphrag interface is used, using the local ollama to provide the LLM interface.Support for using the pip installation
  ☆143Updated 5 months ago
• foldl / chatllm.cpp
  Pure C++ implementation of several models for real-time chatting on your computer (CPU & GPU)
  ☆558Updated this week
• tc-mb / ollama
  Get up and running with Llama 3, Mistral, Gemma, and other large language models.
  ☆26Updated this week
• tc-mb / llama.cpp
  Port of Facebook's LLaMA model in C/C++
  ☆92Updated this week
• limcheekin / open-text-embeddings
  Open Source Text Embedding Models with OpenAI Compatible API
  ☆150Updated 8 months ago
• vtuber-plan / olah
  Self-hosted huggingface mirror service. 自建huggingface镜像服务。
  ☆146Updated 2 weeks ago
• chuanruihu / Level-Navi-Agent-Search
  The Level-Navi Agent, a framework that requires no training and utilizes large language models for deep query understanding and precise s…
  ☆78Updated 3 months ago
• HanzlaJavaid / Free-Search
  Free Search is a wrapper on top of publicly available SearXNG instances to give free internet access as a rest API.
  ☆14Updated last week
• modelscope / modelscope-studio
  A third-party component library based on Gradio.
  ☆92Updated this week
• ModelCloud / GPTQModel
  Production ready LLM model compression/quantization toolkit with hw accelerated inference support for both cpu/gpu via HF, vLLM, and SGLa…
  ☆384Updated this week
• asprenger / ray_vllm_inference
  A simple service that integrates vLLM with Ray Serve for fast and scalable LLM serving.
  ☆65Updated 11 months ago
• matteoserva / GraphLLM
  ☆197Updated 2 weeks ago
• tdrussell / qlora-pipe
  A pipeline parallel training script for LLMs.
  ☆136Updated last week