Alternatives and similar repositories for vllm-openvino

Users that are interested in vllm-openvino are comparing it to the libraries listed below

• slyalin / openvino_devtools
  Tools for easier OpenVINO development/debugging
  ☆9Updated 3 months ago
• openvinotoolkit / openvino.genai
  Run Generative AI models with simple C++/Python API and using OpenVINO Runtime
  ☆295Updated this week
• huggingface / optimum-intel
  🤗 Optimum Intel: Accelerate inference with Intel optimization tools
  ☆473Updated last week
• openvinotoolkit / openvino_tokenizers
  OpenVINO Tokenizers extension
  ☆36Updated last week
• intel / xpumanager
  ☆113Updated 2 months ago
• intel / auto-round
  Advanced Quantization Algorithm for LLMs and VLMs, with support for CPU, Intel GPU, CUDA and HPU. Seamlessly integrated with Torchao, Tra…
  ☆525Updated this week
• opea-project / GenAIComps
  GenAI components at micro-service level; GenAI service composer to create mega-service
  ☆160Updated this week
• wangzhaode / llm-export
  llm-export can export llm model to onnx.
  ☆295Updated 5 months ago
• intel / xFasterTransformer
  ☆427Updated this week
• HabanaAI / vllm-fork
  A high-throughput and memory-efficient inference and serving engine for LLMs
  ☆76Updated this week
• openvinotoolkit / openvino_contrib
  Repository for OpenVINO's extra modules
  ☆129Updated this week
• Hmm466 / OpenVINO-Java-API
  ☆16Updated last year
• openvinotoolkit / npu_compiler
  OpenVINO Intel NPU Compiler
  ☆58Updated last week
• openvinotoolkit / openvino_build_deploy
  Pre-built components and code samples to help you build and deploy production-grade AI applications with the OpenVINO™ Toolkit from Intel
  ☆155Updated this week
• huggingface / tgi-gaudi
  Large Language Model Text Generation Inference on Habana Gaudi
  ☆33Updated 3 months ago
• intel / pti-gpu
  Profiling Tools Interfaces for GPU (PTI for GPU) is a set of Getting Started Documentation and Tools Library to start performance analysi…
  ☆228Updated 3 weeks ago
• huggingface / optimum-benchmark
  🏋️ A unified multi-backend utility for benchmarking Transformers, Timm, PEFT, Diffusers and Sentence-Transformers with full support of O…
  ☆304Updated 3 weeks ago
• intel / onnxruntime
  ONNX Runtime: cross-platform, high performance scoring engine for ML models
  ☆65Updated this week
• ModelCloud / GPTQModel
  Production ready LLM model compression/quantization toolkit with hw accelerated inference support for both cpu/gpu via HF, vLLM, and SGLa…
  ☆633Updated this week
• intel / intel-npu-acceleration-library
  Intel® NPU Acceleration Library
  ☆680Updated 2 months ago
• intel / neural-speed
  An innovative library for efficient LLM inference via low-bit quantization
  ☆349Updated 9 months ago
• Ascend / pytorch
  Ascend PyTorch adapter (torch_npu). Mirror of https://gitee.com/ascend/pytorch
  ☆377Updated this week
• openvinotoolkit / awesome-openvino
  A curated list of OpenVINO based AI projects
  ☆138Updated 2 weeks ago
• vllm-project / llm-compressor
  Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM
  ☆1,518Updated this week
• ROCm / vllm
  A high-throughput and memory-efficient inference and serving engine for LLMs
  ☆84Updated this week
• intel / ai-reference-models
  Intel® AI Reference Models: contains Intel optimizations for running deep learning workloads on Intel® Xeon® Scalable processors and Inte…
  ☆718Updated this week
• triton-inference-server / perf_analyzer
  ☆81Updated last week
• huggingface / optimum-habana
  Easy and lightning fast training of 🤗 Transformers on Habana Gaudi processor (HPU)
  ☆188Updated this week
• openvinotoolkit / openvino_xai
  OpenVINO™ Explainable AI (XAI) Toolkit: Visual Explanation for OpenVINO Models
  ☆32Updated 3 months ago
• openvinotoolkit / openvino_testdrive
  With OpenVINO Test Drive, users can run large language models (LLMs) and models trained by Intel Geti on their devices, including AI PCs …
  ☆26Updated 2 weeks ago