Alternatives and similar repositories for ChatBotApp

Users that are interested in ChatBotApp are comparing it to the libraries listed below

• xxxxyu / FlexNN
  Code for ACM MobiCom 2024 paper "FlexNN: Efficient and Adaptive DNN Inference on Memory-Constrained Edge Devices"
  ☆57Updated 10 months ago
• jeffzhou2000 / ggml-hexagon
  the original reference implementation of a specified llama.cpp backend for Qualcomm Hexagon NPU on Android phone, https://github.com/ggml…
  ☆35Updated 5 months ago
• powerserve-project / PowerServe
  High-speed and easy-use LLM serving framework for local deployment
  ☆139Updated 4 months ago
• jeho-lee / Awesome-On-Device-AI-Systems
  ☆101Updated 3 weeks ago
• UbiquitousLearning / mllm
  Fast Multimodal LLM on Mobile Devices
  ☆1,277Updated last week
• chraac / llama.cpp
  LLM inference in C/C++
  ☆48Updated this week
• MollySophia / rwkv-qualcomm
  Inference RWKV v5, v6 and v7 with Qualcomm AI Engine Direct SDK
  ☆88Updated 2 weeks ago
• quic / ai-engine-direct-helper
  QAI AppBuilder is designed to help developers easily execute models on WoS and Linux platforms. It encapsulates the Qualcomm® AI Runtime …
  ☆96Updated this week
• ARM-software / kleidiai
  This repository is a read-only mirror of https://gitlab.arm.com/kleidi/kleidiai
  ☆106Updated last week
• saic-fi / MobileQuant
  [EMNLP Findings 2024] MobileQuant: Mobile-friendly Quantization for On-device Language Models
  ☆68Updated last year
• stevelaskaridis / awesome-mobile-llm
  Awesome Mobile LLMs
  ☆282Updated 3 weeks ago
• yinuotxie / Efficient-LLM-Inferencing-on-GPUs
  Penn CIS 5650 (GPU Programming and Architecture) Final Project
  ☆44Updated 2 years ago
• Kyrie-Zhao / awesome-real-time-AI
  This is a list of awesome edgeAI inference related papers.
  ☆97Updated 2 years ago
• UbiquitousLearning / Mandheling-DSP-Training
  The open-source project for "Mandheling: Mixed-Precision On-Device DNN Training with DSP Offloading"[MobiCom'2022]
  ☆19Updated 3 years ago
• harleyszhang / llm_counts
  llm theoretical performance analysis tools and support params, flops, memory and latency analysis.
  ☆113Updated 5 months ago
• MegEngine / mperf
  mperf是一个面向移动/嵌入式平台的算子性能调优工具箱
  ☆192Updated 2 years ago
• mlc-ai / relax
  ☆171Updated last week
• quic / ai-hub-apps
  The Qualcomm® AI Hub apps are a collection of state-of-the-art machine learning models optimized for performance (latency, memory etc.) a…
  ☆350Updated this week
• infinigence / SpecEE
  Repo for SpecEE: Accelerating Large Language Model Inference with Speculative Early Exiting (ISCA25)
  ☆68Updated 7 months ago
• InfiniTensor / RefactorGraph
  分层解耦的深度学习推理引擎
  ☆78Updated 10 months ago
• xlite-dev / HGEMM
  ⚡️Write HGEMM from scratch using Tensor Cores with WMMA, MMA and CuTe API, Achieve Peak⚡️ Performance.
  ☆137Updated 7 months ago
• caiwanxianhust / FasterLLaMA
  使用 CUDA C++ 实现的 llama 模型推理框架
  ☆62Updated last year
• DeepLink-org / DLSlime
  DLSlime: Flexible & Efficient Heterogeneous Transfer Toolkit
  ☆84Updated this week
• flagos-ai / FlagCX
  ☆135Updated last week
• LLMServe / SwiftTransformer
  High performance Transformer implementation in C++.
  ☆146Updated 11 months ago
• haozixu / llama.cpp-npu
  ☆39Updated this week
• CodeLinaro / llama.cpp
  LLM inference in C/C++
  ☆20Updated last month
• PKU-SEC-Lab / HybriMoE
  [DAC'25] Official implement of "HybriMoE: Hybrid CPU-GPU Scheduling and Cache Management for Efficient MoE Inference"
  ☆94Updated 6 months ago
• feifeibear / LLMRoofline
  Compare different hardware platforms via the Roofline Model for LLM inference tasks.
  ☆119Updated last year
• infinigence / Semi-PD
  A prefill & decode disaggregated LLM serving framework with shared GPU memory and fine-grained compute isolation.
  ☆119Updated 7 months ago