Alternatives and similar repositories for llm-scaler

Users that are interested in llm-scaler are comparing it to the libraries listed below

• SearchSavior / OpenArc
  Inference engine for Intel devices. Serve LLMs, VLMs, Whisper, Kokoro-TTS, Embedding and Rerank models over OpenAI endpoints.
  ☆261Updated last week
• huawei-csl / SINQ
  Welcome to the official repository of SINQ! A novel, fast and high-quality quantization method designed to make any Large Language Model …
  ☆583Updated this week
• onnx / turnkeyml
  No-code CLI designed for accelerating ONNX workflows
  ☆221Updated 6 months ago
• lemonade-sdk / llamacpp-rocm
  Fresh builds of llama.cpp with AMD ROCm™ 7 acceleration
  ☆139Updated last week
• intel / intel-ai-assistant-builder
  Intel® AI Assistant Builder
  ☆136Updated this week
• perk11 / large-model-proxy
  Run multiple resource-heavy Large Models (LM) on the same machine with limited amount of VRAM/other resources by exposing them on differe…
  ☆85Updated this week
• ROCm / TheRock
  The HIP Environment and ROCm Kit - A lightweight open source build system for HIP and ROCm
  ☆641Updated this week
• projektjoe / GPT-OSS
  From-scratch implementation of OpenAI's GPT-OSS model in Python. No Torch, No GPUs.
  ☆107Updated last month
• devnen / Kitten-TTS-Server
  Self-host the ultra-lightweight Kitten TTS model with this enhanced API server with an intuitive Web UI, large text processing for audiob…
  ☆217Updated 4 months ago
• conanhujinming / text_dedup
  High-Performance Text Deduplication Toolkit
  ☆59Updated 3 months ago
• FastFlowLM / FastFlowLM
  Run LLMs on AMD Ryzen™ AI NPUs in minutes. Just like Ollama - but purpose-built and deeply optimized for the AMD NPUs.
  ☆533Updated last week
• Thireus / GGUF-Tool-Suite
  Input your VRAM and RAM and the toolchain will produce a GGUF model tuned to your system within seconds — flexible model sizing and lowes…
  ☆68Updated this week
• openvinotoolkit / awesome-openvino
  A curated list of OpenVINO based AI projects
  ☆173Updated 5 months ago
• divagr18 / memlayer
  Plug-and-play memory for LLMs in 3 lines of code. Add persistent, intelligent, human-like memory and recall to any model in minutes.
  ☆238Updated 3 weeks ago
• monkesearch / monkeSearch
  fully local, temporally aware natural language file search on your pc! even without a GPU. find relevant files using natural language i…
  ☆158Updated this week
• spectral-compute / scale-examples
  ☆64Updated last year
• inferx-net / inferx
  InferX: Inference as a Service Platform
  ☆143Updated this week
• NimbleEdge / sparse_transformers
  Sparse Inferencing for transformer based LLMs
  ☆215Updated 4 months ago
• theroyallab / YALS
  ☆88Updated last week
• electroglyph / quant_clone
  Generate a llama-quantize command to copy the quantization parameters of any GGUF
  ☆28Updated 4 months ago
• AMD-AGI / Instella
  Fully Open Language Models with Stellar Performance
  ☆310Updated last month
• lhl / strix-halo-testing
  ☆159Updated last month
• iuliaturc / gguf-docs
  Docs for GGUF quantization (unofficial)
  ☆333Updated 5 months ago
• taylorwilsdon / reddacted
  reddacted lets you analyze & sanitize your online footprint using LLMs, PII detection & sentiment analysis to identify anything that migh…
  ☆113Updated 4 months ago
• amd / ryzen-ai-documentation
  Onboarding documentation source for the AMD Ryzen™ AI Software Platform. The AMD Ryzen™ AI Software Platform enables developers to take…
  ☆88Updated this week
• ubergarm / ik_llama.cpp
  llama.cpp fork with additional SOTA quants and improved performance
  ☆21Updated this week
• kalavai-net / kalavai-client
  A platform to self-host AI on easy mode
  ☆181Updated last week
• nomic-ai / llama.cpp
  llama.cpp fork used by GPT4All
  ☆55Updated 10 months ago
• aikitoria / open-gpu-kernel-modules
  NVIDIA Linux open GPU with P2P support
  ☆95Updated 2 weeks ago
• open-edge-platform / edge-developer-kit-reference-scripts
  Developer kits reference setup scripts for various kinds of Intel platforms and GPUs
  ☆39Updated last week