Alternatives and similar repositories for awesome-mobile-llm:

Users that are interested in awesome-mobile-llm are comparing it to the libraries listed below

• UbiquitousLearning / SLM_Survey
  ☆80Updated 3 months ago
• intel / auto-round
  Advanced Quantization Algorithm for LLMs/VLMs.
  ☆364Updated this week
• saic-fi / MobileQuant
  [EMNLP Findings 2024] MobileQuant: Mobile-friendly Quantization for On-device Language Models
  ☆54Updated 4 months ago
• brave-experiments / MELT-public
  codebase for "MELTing Point: Mobile Evaluation of Language Transformers"
  ☆16Updated 6 months ago
• pprp / Awesome-LLM-Quantization
  Awesome list for LLM quantization
  ☆160Updated last month
• SqueezeAILab / TinyAgent
  [EMNLP 2024 Demo] TinyAgent: Function Calling at the Edge!
  ☆354Updated 4 months ago
• wejoncy / QLLM
  A general 2-8 bits quantization toolbox with GPTQ/AWQ/HQQ/VPTQ, and export to onnx/onnx-runtime easily.
  ☆155Updated last week
• facebookresearch / LayerSkip
  Code for "LayerSkip: Enabling Early Exit Inference and Self-Speculative Decoding", ACL 2024
  ☆262Updated 3 weeks ago
• UbiquitousLearning / PhoneLM
  ☆36Updated 2 months ago
• OpenGVLab / EfficientQAT
  EfficientQAT: Efficient Quantization-Aware Training for Large Language Models
  ☆241Updated 3 months ago
• Macaronlin / LLaMA3-Quantization
  A repository dedicated to evaluating the performance of quantizied LLaMA3 using various quantization methods..
  ☆175Updated 2 weeks ago
• efeslab / fiddler
  [ICLR'25] Fast Inference of MoE Models with CPU-GPU Orchestration
  ☆180Updated 2 months ago
• project-etalon / etalon
  LLM Serving Performance Evaluation Harness
  ☆66Updated 5 months ago
• neuralmagic / guidellm
  Evaluate and Enhance Your LLM Deployments for Real-World Inference Needs
  ☆185Updated last month
• NVlabs / MaskLLM
  [NeurIPS 24 Spotlight] MaskLLM: Learnable Semi-structured Sparsity for Large Language Models
  ☆150Updated 3 weeks ago
• hkproj / quantization-notes
  Notes on quantization in neural networks
  ☆66Updated last year
• facebookresearch / SpinQuant
  Code repo for the paper "SpinQuant LLM quantization with learned rotations"
  ☆207Updated 2 months ago
• SqueezeAILab / KVQuant
  [NeurIPS 2024] KVQuant: Towards 10 Million Context Length LLM Inference with KV Cache Quantization
  ☆327Updated 5 months ago
• GreenBitAI / green-bit-llm
  A toolkit for fine-tuning, inferencing, and evaluating GreenBitAI's LLMs.
  ☆80Updated last week
• neuralmagic / compressed-tensors
  A safetensors extension to efficiently store sparse quantized tensors on disk
  ☆66Updated this week
• hahnyuan / PB-LLM
  PB-LLM: Partially Binarized Large Language Models
  ☆150Updated last year
• hmarkc / parallel-prompt-decoding
  Efficient LLM Inference Acceleration using Prompting
  ☆45Updated 3 months ago
• IsaacRe / vllm-kvcompress
  KV cache compression for high-throughput LLM inference
  ☆109Updated last week
• MDK8888 / vllmini
  A minimal implementation of vllm.
  ☆33Updated 6 months ago
• CASE-Lab-UMD / Unified-MoE-Compression
  The official implementation of the paper "Demystifying the Compression of Mixture-of-Experts Through a Unified Framework".
  ☆54Updated 3 months ago
• mani-kantap / llm-inference-solutions
  A collection of all available inference solutions for the LLMs
  ☆76Updated 4 months ago
• TianjinYellow / EdgeDeviceLLMCompetition-Starting-Kit
  ☆41Updated 3 months ago
• mlc-ai / llm-perf-bench
  ☆116Updated 9 months ago
• astramind-ai / Mixture-of-depths
  Unofficial implementation for the paper "Mixture-of-Depths: Dynamically allocating compute in transformer-based language models"
  ☆145Updated 7 months ago
• microsoft / VPTQ
  VPTQ, A Flexible and Extreme low-bit quantization algorithm
  ☆572Updated last week