Alternatives and similar repositories for llama.cpp-npu

Users that are interested in llama.cpp-npu are comparing it to the libraries listed below

• xxxxyu / FlexNN
  [MobiCom 24] Efficient and Adaptive DNN inference under changeable memory budgets
  ☆58Updated last year
• haozixu / htp-ops-lib
  Self-implemented NN operators for Qualcomm's Hexagon NPU
  ☆46Updated 4 months ago
• MollySophia / rwkv-qualcomm
  Inference RWKV v5, v6 and v7 with Qualcomm AI Engine Direct SDK
  ☆90Updated last week
• nycu-caslab / TinyTS
  This is the open-source version of TinyTS. The code is dirty so far. We may clean the code in the future.
  ☆19Updated 6 months ago
• Ther-nullptr / circult-eda-mlsys-tinyml-arxiv-daily
  🎓Automatically Update circult-eda-mlsys-tinyml Papers Daily using Github Actions (Update Every 8th hours)
  ☆10Updated this week
• ARM-software / kleidiai
  This repository is a read-only mirror of https://gitlab.arm.com/kleidi/kleidiai
  ☆113Updated this week
• xinetzone / tvm-book
  ☆18Updated 3 weeks ago
• Qualcomm-AI-research / FP8-quantization
  ☆169Updated 2 years ago
• MegEngine / mperf
  mperf是一个面向移动/嵌入式平台的算子性能调优工具箱
  ☆192Updated 2 years ago
• weishengying / tiny-flash-attention
  使用 cutlass 实现 flash-attention 精简版，具有教学意义
  ☆56Updated last year
• mit-han-lab / parallel-computing-tutorial
  ☆177Updated 2 years ago
• LeiWang1999 / tvm_gpu_gemm
  play gemm with tvm
  ☆92Updated 2 years ago
• tpoisonooo / chgemm
  symmetric int8 gemm
  ☆66Updated 5 years ago
• Qualcomm-AI-research / transformer-quantization
  ☆208Updated 4 years ago
• OpenBitSys / BitDistiller
  [ACL 2024] A novel QAT with Self-Distillation framework to enhance ultra low-bit LLMs.
  ☆134Updated last year
• JackonYang / hands-on-tvm
  hands on model tuning with TVM and profile it on a Mac M1, x86 CPU, and GTX-1080 GPU.
  ☆49Updated 2 years ago
• JieRen98 / SGEMM-SASS-Annotation
  ☆21Updated 4 years ago
• ByteDance-Seed / decoupleQ
  A quantization algorithm for LLM
  ☆148Updated last year
• NVIDIA / online-softmax
  Benchmark code for the "Online normalizer calculation for softmax" paper
  ☆105Updated 7 years ago
• HuangShiqing / LearnAndTry
  ☆19Updated last month
• carlushuang / cpu_gemm_opt
  how to design cpu gemm on x86 with avx256, that can beat openblas.
  ☆73Updated 6 years ago
• jeffzhou2000 / ggml-hexagon
  the original reference implementation of a specified llama.cpp backend for Qualcomm Hexagon NPU on Android phone, https://github.com/ggml…
  ☆35Updated 6 months ago
• INT-FlashAttention2024 / INT-FlashAttention
  ☆85Updated last year
• wangsiping97 / FastGEMV
  High-speed GEMV kernels, at most 2.7x speedup compared to pytorch baseline.
  ☆127Updated last year
• yinuotxie / Efficient-LLM-Inferencing-on-GPUs
  Penn CIS 5650 (GPU Programming and Architecture) Final Project
  ☆44Updated 2 years ago
• xiaoyi018 / simple_gemm
  ☆22Updated 4 years ago
• pigirons / conv3x3_m1
  This is a demo how to write a high performance convolution run on apple silicon
  ☆57Updated 4 years ago
• naver-aics / lut-gemm
  ☆83Updated last year
• microsoft / SparTA
  ☆164Updated last year
• gesanqiu / Chinese_MobileBert_on_SNPE
  Run Chinese MobileBert model on SNPE.
  ☆15Updated 2 years ago