stevelaskaridis / awesome-mobile-llmLinks
Awesome Mobile LLMs
☆288Updated last month
Alternatives and similar repositories for awesome-mobile-llm
Users that are interested in awesome-mobile-llm are comparing it to the libraries listed below
Sorting:
- ☆66Updated last year
- 🎯Accuracy-first quantization toolkit for LLMs, focusing on minimizing quality degradation across Weight Only Quantization, MXFP4, NVFP4,…☆793Updated this week
- [EMNLP Findings 2024] MobileQuant: Mobile-friendly Quantization for On-device Language Models☆68Updated last year
- TinyChatEngine: On-Device LLM Inference Library☆936Updated last year
- Fast Multimodal LLM on Mobile Devices☆1,320Updated this week
- High-speed and easy-use LLM serving framework for local deployment☆141Updated 5 months ago
- Code repo for the paper "SpinQuant LLM quantization with learned rotations"☆367Updated 10 months ago
- The homepage of OneBit model quantization framework.☆197Updated 11 months ago
- An innovative library for efficient LLM inference via low-bit quantization☆351Updated last year
- [ICLR-2025-SLLM Spotlight 🔥]MobiLlama : Small Language Model tailored for edge devices☆669Updated 8 months ago
- llama3.cuda is a pure C/CUDA implementation for Llama 3 model.☆348Updated 8 months ago
- A repository dedicated to evaluating the performance of quantizied LLaMA3 using various quantization methods..☆198Updated 11 months ago
- Code for "LayerSkip: Enabling Early Exit Inference and Self-Speculative Decoding", ACL 2024☆351Updated 8 months ago
- [ACL 2025 Main] EfficientQAT: Efficient Quantization-Aware Training for Large Language Models☆323Updated last month
- Awesome list for LLM quantization☆375Updated 3 months ago
- LLM Inference on consumer devices☆128Updated 9 months ago
- VPTQ, A Flexible and Extreme low-bit quantization algorithm☆671Updated 8 months ago
- ☆41Updated 9 months ago
- This repository contains the training code of ParetoQ introduced in our work "ParetoQ Scaling Laws in Extremely Low-bit LLM Quantization"☆116Updated 2 months ago
- 1.58 Bit LLM on Apple Silicon using MLX☆234Updated last year
- This repository is a read-only mirror of https://gitlab.arm.com/kleidi/kleidiai☆111Updated 2 weeks ago
- A family of compressed models obtained via pruning and knowledge distillation☆362Updated 2 months ago
- For releasing code related to compression methods for transformers, accompanying our publications☆454Updated 11 months ago
- 🏋️ A unified multi-backend utility for benchmarking Transformers, Timm, PEFT, Diffusers and Sentence-Transformers with full support of O…☆325Updated 3 months ago
- On-device LLM Inference Powered by X-Bit Quantization☆274Updated last week
- ☆219Updated 11 months ago
- Low-bit LLM inference on CPU/NPU with lookup table☆906Updated 7 months ago
- ☆576Updated last year
- [ICML 2024] BiLLM: Pushing the Limit of Post-Training Quantization for LLMs☆229Updated last year
- Efficient LLM Inference over Long Sequences☆394Updated 6 months ago