Alternatives and similar repositories for llama-box:

Users that are interested in llama-box are comparing it to the libraries listed below

• gpustack / vox-box
  A text-to-speech and speech-to-text server compatible with the OpenAI API, supporting Whisper, FunASR, Bark, and CosyVoice backends.
  ☆63Updated last month
• gpustack / gguf-parser-go
  Review/Check GGUF files and estimate the memory usage and maximum tokens per second.
  ☆127Updated last week
• leafspark / AutoGGUF
  automatically quant GGUF models
  ☆160Updated this week
• chu-tianxiang / vllm-gptq
  A high-throughput and memory-efficient inference and serving engine for LLMs
  ☆130Updated 8 months ago
• limcheekin / open-text-embeddings
  Open Source Text Embedding Models with OpenAI Compatible API
  ☆147Updated 8 months ago
• horus-ai-labs / DistillFlow
  ☆136Updated 3 weeks ago
• vtuber-plan / olah
  Self-hosted huggingface mirror service. 自建huggingface镜像服务。
  ☆136Updated this week
• wade1010 / graphrag-ui
  The latest graphrag interface is used, using the local ollama to provide the LLM interface.Support for using the pip installation
  ☆142Updated 5 months ago
• THUDM / GLM-Edge
  GLM Series Edge Models
  ☆130Updated 3 weeks ago
• OpenBMB / ollama
  Get up and running with Llama 3, Mistral, Gemma, and other large language models.
  ☆26Updated this week
• microsoft / WizardLM2
  ☆59Updated 10 months ago
• tdrussell / qlora-pipe
  A pipeline parallel training script for LLMs.
  ☆128Updated last month
• matatonic / openedai-vision
  An OpenAI API compatible API for chat with image input and questions about the images. aka Multimodal.
  ☆231Updated last week
• nvwacloud / tensorlink
  Unlock Unlimited Potential! Share Your GPU Power Across Your Local Network!
  ☆48Updated 8 months ago
• modelscope / dash-infer
  DashInfer is a native LLM inference engine aiming to deliver industry-leading performance atop various hardware architectures, including …
  ☆237Updated this week
• chuanruihu / Level-Navi-Agent-Search
  The Level-Navi Agent, a framework that requires no training and utilizes large language models for deep query understanding and precise s…
  ☆66Updated 2 months ago
• ikawrakow / ik_llama.cpp
  llama.cpp fork with additional SOTA quants and improved performance
  ☆202Updated this week
• inferflow / inferflow
  Inferflow is an efficient and highly configurable inference engine for large language models (LLMs).
  ☆237Updated 11 months ago
• 01-ai / Descartes
  ☆107Updated 11 months ago
• asprenger / ray_vllm_inference
  A simple service that integrates vLLM with Ray Serve for fast and scalable LLM serving.
  ☆63Updated 11 months ago
• npuichigo / openai_trtllm
  OpenAI compatible API for TensorRT LLM triton backend
  ☆200Updated 7 months ago
• etalab-ia / albert-models
  Deployment a light and full OpenAI API for production with vLLM to support /v1/embeddings with all embeddings models.
  ☆41Updated 7 months ago
• OpenGVLab / EfficientQAT
  EfficientQAT: Efficient Quantization-Aware Training for Large Language Models
  ☆250Updated 5 months ago
• AUGMXNT / deccp
  Evaling and unaligning Chinese LLM censorship
  ☆57Updated 5 months ago
• thunlp / Delta-CoMe
  Delta-CoMe can achieve near loss-less 1-bit compressin which has been accepted by NeurIPS 2024
  ☆54Updated 3 months ago
• IEIT-Yuan / Yuan2.0-M32
  Mixture-of-Experts (MoE) Language Model
  ☆185Updated 6 months ago
• shell-nlp / gpt_server
  gpt_server是一个用于生产级部署LLMs或Embedding的开源框架。
  ☆158Updated this week