Alternatives and similar repositories for airllm

Users that are interested in airllm are comparing it to the libraries listed below

• InternLM / lmdeploy
  LMDeploy is a toolkit for compressing, deploying, and serving LLMs.
  ☆7,606Updated this week
• axolotl-ai-cloud / axolotl
  Go ahead and axolotl questions
  ☆11,251Updated last week
• AutoGPTQ / AutoGPTQ
  An easy-to-use LLMs quantization package with user-friendly apis, based on GPTQ algorithm.
  ☆5,028Updated 10 months ago
• predibase / lorax
  Multi-LoRA inference server that scales to 1000s of fine-tuned LLMs
  ☆3,718Updated 8 months ago
• arcee-ai / mergekit
  Tools for merging pretrained large language models.
  ☆6,783Updated 2 weeks ago
• RahulSChand / gpu_poor
  Calculate token/s & GPU memory requirement for any LLM. Supports llama.cpp/ggml/bnb/QLoRA quantization
  ☆1,389Updated last year
• turboderp-org / exllamav2
  A fast inference library for running LLMs locally on modern consumer-class GPUs
  ☆4,440Updated 2 months ago
• huggingface / text-generation-inference
  Large Language Model Text Generation Inference
  ☆10,757Updated last month
• Tiiny-AI / PowerInfer
  High-speed Large Language Model Serving for Local Deployment
  ☆8,635Updated 2 weeks ago
• QwenLM / Qwen-Agent
  Agent framework and applications built upon Qwen>=3.0, featuring Function Calling, MCP, Code Interpreter, RAG, Chrome extension, etc.
  ☆13,234Updated last week
• InternLM / xtuner
  A Next-Generation Training Engine Built for Ultra-Large MoE Models
  ☆5,082Updated last week
• imoneoi / openchat
  OpenChat: Advancing Open-source Language Models with Imperfect Data
  ☆5,472Updated last year
• artidoro / qlora
  QLoRA: Efficient Finetuning of Quantized LLMs
  ☆10,835Updated last year
• ModelTC / LightLLM
  LightLLM is a Python-based LLM (Large Language Model) inference and serving framework, notable for its lightweight design, easy scalabili…
  ☆3,888Updated this week
• huggingface / text-embeddings-inference
  A blazing fast inference solution for text embeddings models
  ☆4,476Updated last week
• abetlen / llama-cpp-python
  Python bindings for llama.cpp
  ☆9,971Updated 5 months ago
• sgl-project / sglang
  SGLang is a high-performance serving framework for large language models and multimodal models.
  ☆23,439Updated this week
• huggingface / autotrain-advanced
  🤗 AutoTrain Advanced
  ☆4,555Updated 2 weeks ago
• casper-hansen / AutoAWQ
  AutoAWQ implements the AWQ algorithm for 4-bit quantization with a 2x speedup during inference. Documentation:
  ☆2,314Updated 9 months ago
• meta-pytorch / torchtune
  PyTorch native post-training library
  ☆5,669Updated this week
• FlagOpen / FlagEmbedding
  Retrieval and Retrieval-augmented LLMs
  ☆11,280Updated last month
• meta-llama / llama-cookbook
  Welcome to the Llama Cookbook! This is your go to guide for Building with Llama: Getting started with Inference, Fine-Tuning, RAG. We als…
  ☆18,190Updated 3 months ago
• bentoml / OpenLLM
  Run any open-source LLMs, such as DeepSeek and Llama, as OpenAI compatible API endpoint in the cloud.
  ☆12,099Updated 2 weeks ago
• mit-han-lab / streaming-llm
  [ICLR 2024] Efficient Streaming Language Models with Attention Sinks
  ☆7,182Updated last year
• mit-han-lab / llm-awq
  [MLSys 2024 Best Paper Award] AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration
  ☆3,436Updated 6 months ago
• Lightning-AI / litgpt
  20+ high-performance LLMs with recipes to pretrain, finetune and deploy at scale.
  ☆13,155Updated this week
• microsoft / LLMLingua
  [EMNLP'23, ACL'24] To speed up LLMs' inference and enhance LLM's perceive of key information, compress the prompt and KV-Cache, which ach…
  ☆5,823Updated 3 months ago
• ggml-org / ggml
  Tensor library for machine learning
  ☆13,923Updated this week
• bitsandbytes-foundation / bitsandbytes
  Accessible large language models via k-bit quantization for PyTorch.
  ☆7,939Updated 3 weeks ago
• lm-sys / RouteLLM
  A framework for serving and evaluating LLM routers - save LLM costs without compromising quality
  ☆4,581Updated last year