Alternatives and similar repositories for chatllm.cpp

Users that are interested in chatllm.cpp are comparing it to the libraries listed below

• intel / neural-speed
  An innovative library for efficient LLM inference via low-bit quantization
  ☆352Updated last year
• intel / auto-round
  🎯An accuracy-first, highly efficient quantization toolkit for LLMs, designed to minimize quality degradation across Weight-Only Quantiza…
  ☆839Updated this week
• ikawrakow / ik_llama.cpp
  llama.cpp fork with additional SOTA quants and improved performance
  ☆1,587Updated this week
• gpustack / llama-box
  LM inference server implementation based on *.cpp.
  ☆295Updated 2 months ago
• Maximilian-Winter / llama-cpp-agent
  The llama-cpp-agent framework is a tool designed for easy interaction with Large Language Models (LLMs). Allowing users to chat with LLM …
  ☆615Updated 11 months ago
• RWKV / rwkv.cpp
  INT4/INT5/INT8 and FP16 inference on CPU for RWKV language model
  ☆1,563Updated 10 months ago
• microsoft / T-MAC
  Low-bit LLM inference on CPU/NPU with lookup table
  ☆916Updated 8 months ago
• gpustack / gguf-parser-go
  Review/Check GGUF files and estimate the memory usage and maximum tokens per second.
  ☆238Updated last month
• dropbox / hqq
  Official implementation of Half-Quadratic Quantization (HQQ)
  ☆912Updated last month
• ModelCloud / GPTQModel
  LLM model quantization (compression) toolkit with hw acceleration support for Nvidia CUDA, AMD ROCm, Intel XPU and Intel/AMD/Apple CPU vi…
  ☆1,007Updated this week
• lapp0 / lm-inference-engines
  Comparison of Language Model Inference Engines
  ☆239Updated last year
• monatis / clip.cpp
  CLIP inference in plain C/C++ with no extra dependencies
  ☆549Updated 7 months ago
• turboderp-org / exllamav3
  An optimized quantization and inference library for running LLMs locally on modern consumer-class GPUs
  ☆626Updated last week
• leafspark / AutoGGUF
  automatically quant GGUF models
  ☆219Updated last month
• andrewkchan / deepseek.cpp
  CPU inference for the DeepSeek family of large language models in C++
  ☆317Updated 4 months ago
• skeskinen / bert.cpp
  ggml implementation of BERT
  ☆498Updated last year
• aphrodite-engine / aphrodite-engine
  Large-scale LLM inference engine
  ☆1,641Updated 2 weeks ago
• PABannier / bark.cpp
  Suno AI's Bark model in C/C++ for fast text-to-speech generation
  ☆854Updated last year
• microsoft / VPTQ
  VPTQ, A Flexible and Extreme low-bit quantization algorithm
  ☆674Updated 9 months ago
• QwenLM / qwen.cpp
  C++ implementation of Qwen-LM
  ☆616Updated last year
• adriancable / qwen3.c
  Local Qwen3 LLM inference. One easy-to-understand file of C source with no dependencies.
  ☆156Updated 7 months ago
• staghado / vit.cpp
  Inference Vision Transformer (ViT) in plain C/C++ with ggml
  ☆306Updated last year
• ngxson / wllama
  WebAssembly binding for llama.cpp - Enabling on-browser LLM inference
  ☆987Updated last month
• vectorch-ai / ScaleLLM
  A high-performance inference system for large language models, designed for production environments.
  ☆491Updated last month
• ubergarm / r1-ktransformers-guide
  run DeepSeek-R1 GGUFs on KTransformers
  ☆260Updated 11 months ago
• microsoft / onnxruntime-genai
  Generative AI extensions for onnxruntime
  ☆953Updated this week
• likejazz / llama3.cuda
  llama3.cuda is a pure C/CUDA implementation for Llama 3 model.
  ☆350Updated 9 months ago
• abetlen / ggml-python
  Python bindings for ggml
  ☆147Updated last year
• mlc-ai / tokenizers-cpp
  Universal cross-platform tokenizers binding to HF and sentencepiece
  ☆451Updated 2 weeks ago
• mit-han-lab / TinyChatEngine
  TinyChatEngine: On-Device LLM Inference Library
  ☆941Updated last year