Alternatives and similar repositories for llm-scaler

Users that are interested in llm-scaler are comparing it to the libraries listed below

• perk11 / large-model-proxy
  Run multiple resource-heavy Large Models (LM) on the same machine with limited amount of VRAM/other resources by exposing them on differe…
  ☆82Updated last week
• SearchSavior / OpenArc
  Lightweight Inference server for OpenVINO
  ☆211Updated this week
• intel / intel-ai-assistant-builder
  Intel® AI Assistant Builder
  ☆106Updated this week
• qrv0 / llm-ripper
  LLM Ripper is a framework for component extraction (embeddings, attention heads, FFNs), activation capture, functional analysis, and adap…
  ☆46Updated this week
• onnx / turnkeyml
  No-code CLI designed for accelerating ONNX workflows
  ☆214Updated 3 months ago
• crashr / gppm
  GPU Power and Performance Manager
  ☆61Updated 11 months ago
• l33tkr3w / LlamaCards
  LlamaCards is a web application that provides a dynamic interface for interacting with LLM models in real-time. This app allows users to …
  ☆39Updated last year
• grctest / Electron-BitNet
  Running Microsoft's BitNet via Electron, React & Astro
  ☆44Updated 3 months ago
• cp3249 / splaa
  SPLAA is an AI assistant framework that utilizes voice recognition, text-to-speech, and tool-calling capabilities to provide a conversati…
  ☆28Updated 4 months ago
• jukofyork / transplant-vocab
  Transplants vocabulary between language models, enabling the creation of draft models for speculative decoding WITHOUT retraining.
  ☆42Updated 2 weeks ago
• aikitoria / open-gpu-kernel-modules
  NVIDIA Linux open GPU with P2P support
  ☆54Updated this week
• abgulati / hf-waitress
  Serving LLMs in the HF-Transformers format via a PyFlask API
  ☆71Updated last year
• chigkim / Ollama-MMLU-Pro
  ☆100Updated last month
• NimbleEdge / sparse_transformers
  Sparse Inferencing for transformer based LLMs
  ☆198Updated last month
• Thireus / GGUF-Tool-Suite
  Input your VRAM and RAM and the toolchain will produce a GGUF model tuned to your system within seconds — flexible model sizing and lowes…
  ☆45Updated this week
• Toy-97 / Chat-WebUI
  Chat WebUI is an easy-to-use user interface for interacting with AI, and it comes with multiple useful built-in tools such as web search …
  ☆45Updated 3 weeks ago
• Lanerra / saga
  Autonomous, agentic, creative story writing system that incorporates stored embeddings and Knowledge Graphs.
  ☆79Updated this week
• Rivridis / LLM-Assistant
  Locally running LLM with internet access
  ☆96Updated 2 months ago
• amd / ryzen-ai-documentation
  Onboarding documentation source for the AMD Ryzen™ AI Software Platform. The AMD Ryzen™ AI Software Platform enables developers to take…
  ☆78Updated this week
• conanhujinming / text_dedup
  High-Performance Text Deduplication Toolkit
  ☆56Updated last month
• codelion / ellora
  Enhancing LLMs with LoRA
  ☆137Updated 2 weeks ago
• LAION-AI / Desktop_BUD-E
  BUD-E (Buddy) is an open-source voice assistant framework that facilitates seamless interaction with AI models and APIs, enabling the cre…
  ☆39Updated last year
• Picovoice / picollm
  On-device LLM Inference Powered by X-Bit Quantization
  ☆268Updated last month
• AMD-AGI / Instella
  Fully Open Language Models with Stellar Performance
  ☆247Updated last month
• electroglyph / quant_clone
  Generate a llama-quantize command to copy the quantization parameters of any GGUF
  ☆24Updated last month
• theroyallab / YALS
  ☆85Updated last week
• inferx-net / inferx
  InferX: Inference as a Service Platform
  ☆135Updated this week
• bold84 / cot_proxy
  Smart proxy for LLM APIs that enables model-specific parameter control, automatic mode switching (like Qwen3's /think and /no_think), and…
  ☆50Updated 4 months ago
• stringandstickytape / MaxsAiStudio
  A Windows tool to query various LLM AIs. Supports branched conversations, history and summaries among others.
  ☆33Updated last week
• extopico / llama-server_mcp_proxy
  Simple node proxy for llama-server that enables MCP use
  ☆13Updated 4 months ago