Alternatives and similar repositories for awesome-mobile-llm

Users that are interested in awesome-mobile-llm are comparing it to the libraries listed below

• UbiquitousLearning / mllm
  Fast Multimodal LLM on Mobile Devices
  ☆849Updated last month
• apple / ml-recurrent-drafter
  ☆209Updated 3 months ago
• powerserve-project / PowerServe
  High-speed and easy-use LLM serving framework for local deployment
  ☆103Updated last month
• pprp / Awesome-LLM-Quantization
  Awesome list for LLM quantization
  ☆213Updated 4 months ago
• facebookresearch / SpinQuant
  Code repo for the paper "SpinQuant LLM quantization with learned rotations"
  ☆272Updated 3 months ago
• efeslab / fiddler
  [ICLR'25] Fast Inference of MoE Models with CPU-GPU Orchestration
  ☆209Updated 5 months ago
• Infini-AI-Lab / Sequoia
  scalable and robust tree-based speculative decoding algorithm
  ☆345Updated 3 months ago
• saic-fi / MobileQuant
  [EMNLP Findings 2024] MobileQuant: Mobile-friendly Quantization for On-device Language Models
  ☆56Updated 7 months ago
• microsoft / TransformerCompression
  For releasing code related to compression methods for transformers, accompanying our publications
  ☆427Updated 3 months ago
• NVlabs / Minitron
  A family of compressed models obtained via pruning and knowledge distillation
  ☆336Updated 6 months ago
• intel / auto-round
  Advanced Quantization Algorithm for LLMs/VLMs.
  ☆460Updated this week
• facebookresearch / LayerSkip
  Code for "LayerSkip: Enabling Early Exit Inference and Self-Speculative Decoding", ACL 2024
  ☆294Updated last week
• Aaronhuang-778 / BiLLM
  [ICML 2024] BiLLM: Pushing the Limit of Post-Training Quantization for LLMs
  ☆216Updated 4 months ago
• IsaacRe / vllm-kvcompress
  KV cache compression for high-throughput LLM inference
  ☆126Updated 3 months ago
• UbiquitousLearning / PhoneLM
  ☆56Updated 5 months ago
• IntelLabs / Hardware-Aware-Automated-Machine-Learning
  ☆54Updated 2 weeks ago
• OpenGVLab / EfficientQAT
  EfficientQAT: Efficient Quantization-Aware Training for Large Language Models
  ☆266Updated 7 months ago
• Cornell-RelaxML / qtip
  ☆131Updated last month
• mbzuai-oryx / MobiLlama
  MobiLlama : Small Language Model tailored for edge devices
  ☆636Updated this week
• NexaAI / Awesome-LLMs-on-device
  Awesome LLMs on Device: A Comprehensive Survey
  ☆1,093Updated 4 months ago
• jy-yuan / KIVI
  [ICML 2024] KIVI: A Tuning-Free Asymmetric 2bit Quantization for KV Cache
  ☆295Updated 3 months ago
• mobiusml / gemlite
  Fast low-bit matmul kernels in Triton
  ☆299Updated this week
• andrewkchan / yalm
  Yet Another Language Model: LLM inference in C++/CUDA, no libraries except for I/O
  ☆357Updated 4 months ago
• neuralmagic / nm-vllm
  A high-throughput and memory-efficient inference and serving engine for LLMs
  ☆263Updated 7 months ago
• intel / neural-speed
  An innovative library for efficient LLM inference via low-bit quantization
  ☆350Updated 8 months ago
• Cornell-RelaxML / quip-sharp
  ☆532Updated 6 months ago
• xuyuzhuang11 / OneBit
  The homepage of OneBit model quantization framework.
  ☆176Updated 3 months ago
• Picovoice / picollm
  On-device LLM Inference Powered by X-Bit Quantization
  ☆238Updated this week
• SqueezeAILab / KVQuant
  [NeurIPS 2024] KVQuant: Towards 10 Million Context Length LLM Inference with KV Cache Quantization
  ☆352Updated 9 months ago
• VITA-Group / Q-GaLore
  Q-GaLore: Quantized GaLore with INT4 Projection and Layer-Adaptive Low-Rank Gradients.
  ☆199Updated 9 months ago