gpustack / llama-boxLinks
LM inference server implementation based on *.cpp.
☆203Updated this week
Alternatives and similar repositories for llama-box
Users that are interested in llama-box are comparing it to the libraries listed below
Sorting:
- Review/Check GGUF files and estimate the memory usage and maximum tokens per second.☆173Updated this week
- A text-to-speech and speech-to-text server compatible with the OpenAI API, supporting Whisper, FunASR, Bark, and CosyVoice backends.☆115Updated last week
- xllamacpp - a Python wrapper of llama.cpp☆40Updated last week
- run DeepSeek-R1 GGUFs on KTransformers☆231Updated 3 months ago
- Library for model distillation☆142Updated 3 months ago
- Pure C++ implementation of several models for real-time chatting on your computer (CPU & GPU)☆614Updated this week
- Port of Facebook's LLaMA model in C/C++☆52Updated last month
- ☆88Updated 2 months ago
- CPU inference for the DeepSeek family of large language models in C++☆296Updated this week
- gpt_server是一个用于生产级部署LLMs、Embedding、Reranker、ASR和TTS的开源框架。☆184Updated this week
- automatically quant GGUF models☆179Updated last week
- Mixture-of-Experts (MoE) Language Model☆188Updated 8 months ago
- The latest graphrag interface is used, using the local ollama to provide the LLM interface.Support for using the pip installation☆150Updated 7 months ago
- A high-throughput and memory-efficient inference and serving engine for LLMs☆130Updated 11 months ago
- Port of Facebook's LLaMA model in C/C++☆93Updated 3 weeks ago
- DashInfer is a native LLM inference engine aiming to deliver industry-leading performance atop various hardware architectures, including …☆253Updated this week
- Get up and running with Llama 3, Mistral, Gemma, and other large language models.☆26Updated 3 weeks ago
- 研究GOT-OCR-项目落地加速,不限语言☆60Updated 7 months ago
- ☆53Updated 5 months ago
- Open Source Text Embedding Models with OpenAI Compatible API☆153Updated 10 months ago
- The main repository for building Pascal-compatible versions of ML applications and libraries.☆87Updated 2 weeks ago
- Production ready LLM model compression/quantization toolkit with hw accelerated inference support for both cpu/gpu via HF, vLLM, and SGLa…☆590Updated this week
- Tool to download models from Huggingface Hub and convert them to GGML/GGUF for llama.cpp☆148Updated last month
- GLM Series Edge Models☆141Updated 3 months ago
- Convert files into markdown to help RAG or LLM understand, based on markitdown and MinerU, which could provide high quality pdf parser.☆99Updated 2 months ago
- This is InfiniRetri, a tool enhance Transformer-based LLMs(Large Language Model) ablity to hangle Long-Context.☆108Updated 2 months ago
- Receipts for creating AI Applications with APIs from DashScope (and friends)!☆54Updated 8 months ago
- The LLM API Benchmark Tool is a flexible Go-based utility designed to measure and analyze the performance of OpenAI-compatible API endpoi…☆25Updated 3 months ago
- Comparison of Language Model Inference Engines☆217Updated 5 months ago
- LLM inference in C/C++☆21Updated 2 months ago