Alternatives and similar repositories for llama.cpp

Users that are interested in llama.cpp are comparing it to the libraries listed below

• zai-org / GLM-Edge
  GLM Series Edge Models
  ☆157Updated 7 months ago
• infinigence / Infini-Megrez-Omni
  ☆242Updated 11 months ago
• 1694439208 / GOT-OCR-Inference
  研究GOT-OCR-项目落地加速，不限语言
  ☆62Updated last year
• HimariO / llama.cpp.qwen2.5vl
  Port of Facebook's LLaMA model in C/C++
  ☆67Updated 9 months ago
• Achazwl / mlc
  MiniCPM on Android platform.
  ☆636Updated 10 months ago
• DakeQQ / Native-LLM-for-Android
  Demonstration of running a native LLM on Android device.
  ☆226Updated this week
• OpenSQZ / MiniCPM-V-CookBook
  Cook up amazing multimodal AI applications effortlessly with MiniCPM-o
  ☆290Updated this week
• tc-mb / ollama
  Get up and running with Llama 3, Mistral, Gemma, and other large language models.
  ☆32Updated this week
• infinigence / Infini-Megrez
  ☆341Updated 3 months ago
• DataXujing / Qwen1.5-0.5b-chat-android
  基于MNN-llm的安卓手机部署大语言模型：Qwen1.5-0.5B-Chat
  ☆89Updated last year
• OpenBMB / MiniCPM-CookBook
  This is a user guide for the MiniCPM and MiniCPM-V series of small language models (SLMs) developed by ModelBest. “面壁小钢炮” focuses on achi…
  ☆299Updated 7 months ago
• fyabc / vllm
  A high-throughput and memory-efficient inference and serving engine for LLMs
  ☆46Updated 4 months ago
• QwenLM / qwen.cpp
  C++ implementation of Qwen-LM
  ☆616Updated last year
• wangzhaode / llm-export
  llm-export can export llm model to onnx.
  ☆343Updated 3 months ago
• yvonwin / qwen2.cpp
  qwen2 and llama3 cpp implementation
  ☆49Updated last year
• shootime2021 / APUS-xDAN-4.0-moe
  Its an open source LLM based on MOE Structure.
  ☆58Updated last year
• Oneflow-Inc / diffusers
  ☆31Updated last year
• OpenBMB / MobileCPM
  A Toolkit for Running On-device Large Language Models (LLMs) in APP
  ☆81Updated last year
• wangzhaode / mnn-stable-diffusion
  stable diffusion using mnn
  ☆67Updated 2 years ago
• AXERA-TECH / ax-llm
  Explore LLM model deployment based on AXera's AI chips
  ☆139Updated this week
• zkh2016 / LLMFarm
  MiniCPM on iOS.
  ☆67Updated 10 months ago
• sophgo / ChatGLM3-TPU
  run chatglm3-6b in BM1684X
  ☆39Updated last year
• Tlntin / qwen-ascend-llm
  ☆55Updated last year
• XiaoMi / StableDiffusionOnDevice
  本项目是一个通过文字生成图片的项目，基于开源模型Stable Diffusion V1.5生成可以在手机的CPU和NPU上运行的模型，包括其配套的模型运行框架。
  ☆231Updated last year
• sophgo / LLM-TPU
  Run generative AI models in sophgo BM1684X/BM1688
  ☆266Updated 2 weeks ago
• hpcaitech / SwiftInfer
  Efficient AI Inference & Serving
  ☆479Updated 2 years ago
• viitrix / vt-transformer
  Transformer framework for edge computing based on C++.
  ☆130Updated last year
• mbzuai-oryx / MobiLlama
  [ICLR-2025-SLLM Spotlight 🔥]MobiLlama : Small Language Model tailored for edge devices
  ☆668Updated 8 months ago
• Tencent / AngelSlim
  Model compression toolkit engineered for enhanced usability, comprehensiveness, and efficiency.
  ☆314Updated this week
• sophgo / ChatGLM2-TPU
  run ChatGLM2-6B in BM1684X
  ☆49Updated last year