Alternatives and similar repositories for awesome-mobile-llm

Users that are interested in awesome-mobile-llm are comparing it to the libraries listed below

• powerserve-project / PowerServe
  High-speed and easy-use LLM serving framework for local deployment
  ☆145Updated 5 months ago
• UbiquitousLearning / mllm
  Fast Multimodal LLM on Mobile Devices
  ☆1,370Updated this week
• mit-han-lab / TinyChatEngine
  TinyChatEngine: On-Device LLM Inference Library
  ☆941Updated last year
• saic-fi / MobileQuant
  [EMNLP Findings 2024] MobileQuant: Mobile-friendly Quantization for On-device Language Models
  ☆68Updated last year
• exo-explore / mlx-bitnet
  1.58 Bit LLM on Apple Silicon using MLX
  ☆242Updated last year
• Picovoice / picollm
  On-device LLM Inference Powered by X-Bit Quantization
  ☆278Updated last week
• intel / auto-round
  🎯An accuracy-first, highly efficient quantization toolkit for LLMs, designed to minimize quality degradation across Weight-Only Quantiza…
  ☆839Updated this week
• jeho-lee / Awesome-On-Device-AI-Systems
  ☆116Updated this week
• UbiquitousLearning / PhoneLM
  ☆65Updated last year
• microsoft / TransformerCompression
  For releasing code related to compression methods for transformers, accompanying our publications
  ☆455Updated last year
• lx200916 / ChatBotApp
  ☆42Updated 10 months ago
• pprp / Awesome-LLM-Quantization
  Awesome list for LLM quantization
  ☆384Updated 3 months ago
• microsoft / VPTQ
  VPTQ, A Flexible and Extreme low-bit quantization algorithm
  ☆674Updated 9 months ago
• facebookresearch / MobileLLM
  MobileLLM Optimizing Sub-billion Parameter Language Models for On-Device Use Cases. In ICML 2024.
  ☆1,406Updated 9 months ago
• apple / ml-recurrent-drafter
  ☆219Updated last year
• microsoft / T-MAC
  Low-bit LLM inference on CPU/NPU with lookup table
  ☆915Updated 7 months ago
• Macaronlin / LLaMA3-Quantization
  A repository dedicated to evaluating the performance of quantizied LLaMA3 using various quantization methods..
  ☆198Updated last year
• facebookresearch / SpinQuant
  Code repo for the paper "SpinQuant LLM quantization with learned rotations"
  ☆373Updated 11 months ago
• Cornell-RelaxML / quip-sharp
  ☆576Updated last year
• Infini-AI-Lab / Sequoia
  scalable and robust tree-based speculative decoding algorithm
  ☆366Updated last year
• facebookresearch / LayerSkip
  Code for "LayerSkip: Enabling Early Exit Inference and Self-Speculative Decoding", ACL 2024
  ☆355Updated last week
• ARM-software / kleidiai
  This repository is a read-only mirror of https://gitlab.arm.com/kleidi/kleidiai
  ☆113Updated this week
• efeslab / fiddler
  [ICLR'25] Fast Inference of MoE Models with CPU-GPU Orchestration
  ☆258Updated last year
• google-ai-edge / LiteRT-LM
  ☆786Updated this week
• antimatter15 / reverse-engineering-gemma-3n
  Reverse Engineering Gemma 3n: Google's New Edge-Optimized Language Model
  ☆262Updated 8 months ago
• Aaronhuang-778 / BiLLM
  [ICML 2024] BiLLM: Pushing the Limit of Post-Training Quantization for LLMs
  ☆229Updated last year
• intel / neural-speed
  An innovative library for efficient LLM inference via low-bit quantization
  ☆352Updated last year
• NVIDIA / Star-Attention
  Efficient LLM Inference over Long Sequences
  ☆394Updated 7 months ago
• huggingface / optimum-benchmark
  🏋️ A unified multi-backend utility for benchmarking Transformers, Timm, PEFT, Diffusers and Sentence-Transformers with full support of O…
  ☆327Updated 4 months ago
• xuyuzhuang11 / OneBit
  The homepage of OneBit model quantization framework.
  ☆200Updated 11 months ago