Alternatives and similar repositories for llama.cpp

Users that are interested in llama.cpp are comparing it to the libraries listed below

• jeffzhou2000 / ggml-hexagon
  the original reference implementation of a specified llama.cpp backend for Qualcomm Hexagon NPU on Android phone, https://github.com/ggml…
  ☆35Updated 6 months ago
• MollySophia / rwkv-qualcomm
  Inference RWKV v5, v6 and v7 with Qualcomm AI Engine Direct SDK
  ☆90Updated this week
• chraac / llama-cpp-qnn-builder
  ☆11Updated last week
• quic / ai-engine-direct-helper
  QAI AppBuilder is designed to help developers easily execute models on WoS and Linux platforms. It encapsulates the Qualcomm® AI Runtime …
  ☆111Updated this week
• quic / ai-hub-apps
  The Qualcomm® AI Hub apps are a collection of state-of-the-art machine learning models optimized for performance (latency, memory etc.) a…
  ☆369Updated last week
• wangzhaode / llm-export
  llm-export can export llm model to onnx.
  ☆344Updated 3 months ago
• lx200916 / ChatBotApp
  ☆42Updated 10 months ago
• CodeLinaro / llama.cpp
  LLM inference in C/C++
  ☆20Updated 3 months ago
• futz12 / ncnn_llm
  A repo for llm on ncnn
  ☆189Updated last month
• MollySophia / rwkv-mobile
  Inference RWKV with multiple supported backends.
  ☆77Updated this week
• microsoft / T-MAC
  Low-bit LLM inference on CPU/NPU with lookup table
  ☆916Updated 8 months ago
• XiaoMi / StableDiffusionOnDevice
  本项目是一个通过文字生成图片的项目，基于开源模型Stable Diffusion V1.5生成可以在手机的CPU和NPU上运行的模型，包括其配套的模型运行框架。
  ☆231Updated last year
• wangzhaode / onnx-llm
  llm deploy project based onnx.
  ☆49Updated last year
• powerserve-project / PowerServe
  High-speed and easy-use LLM serving framework for local deployment
  ☆145Updated 6 months ago
• MegEngine / mperf
  mperf是一个面向移动/嵌入式平台的算子性能调优工具箱
  ☆193Updated 2 years ago
• haozixu / htp-ops-lib
  Self-implemented NN operators for Qualcomm's Hexagon NPU
  ☆46Updated 4 months ago
• wangzhaode / mnn-stable-diffusion
  stable diffusion using mnn
  ☆67Updated 2 years ago
• haozixu / llama.cpp-npu
  ☆56Updated last month
• gesanqiu / Chinese_MobileBert_on_SNPE
  Run Chinese MobileBert model on SNPE.
  ☆15Updated 2 years ago
• mlc-ai / tokenizers-cpp
  Universal cross-platform tokenizers binding to HF and sentencepiece
  ☆451Updated 2 weeks ago
• lrw04 / llama2.c-to-ncnn
  A converter for llama2.c legacy models to ncnn models.
  ☆79Updated 2 years ago
• ARM-software / kleidiai
  This repository is a read-only mirror of https://gitlab.arm.com/kleidi/kleidiai
  ☆113Updated last week
• sunkx109 / llama.cpp
  llama 2 Inference
  ☆43Updated 2 years ago
• UbiquitousLearning / mllm
  Fast Multimodal LLM on Mobile Devices
  ☆1,370Updated this week
• DakeQQ / Native-LLM-for-Android
  Demonstration of running a native LLM on Android device.
  ☆226Updated this week
• Repeerc / flash-attention-v2-RDNA3-minimal
  a simple Flash Attention v2 implementation with ROCM (RDNA3 GPU, roc wmma), mainly used for stable diffusion(ComfyUI) in Windows ZLUDA en…
  ☆51Updated last year
• quic / qidk
  ☆181Updated 2 weeks ago
• pzhao-eng / FlashMLA
  ☆61Updated 6 months ago
• MegEngine / InferLLM
  a lightweight LLM model inference framework
  ☆749Updated last year
• TroyTzou / mlc-llm-android
  参考自mlc-llm，个人尝试在android手机上部署大模型并运行
  ☆90Updated last year