Alternatives and similar repositories for vllm-openvino

Users that are interested in vllm-openvino are comparing it to the libraries listed below

• intel / ai-containers
  This repository contains Dockerfiles, scripts, yaml files, Helm charts, etc. used to scale out AI containers with versions of TensorFlow …
  ☆48Updated last week
• openvinotoolkit / openvino.genai
  Run Generative AI models with simple C++/Python API and using OpenVINO Runtime
  ☆303Updated this week
• ROCm / vllm
  A high-throughput and memory-efficient inference and serving engine for LLMs
  ☆85Updated this week
• openvinotoolkit / openvino_tokenizers
  OpenVINO Tokenizers extension
  ☆37Updated this week
• openvinotoolkit / npu_compiler
  OpenVINO Intel NPU Compiler
  ☆60Updated this week
• huggingface / tgi-gaudi
  Large Language Model Text Generation Inference on Habana Gaudi
  ☆34Updated 3 months ago
• openvinotoolkit / awesome-openvino
  A curated list of OpenVINO based AI projects
  ☆141Updated 2 weeks ago
• powerserve-project / PowerServe
  High-speed and easy-use LLM serving framework for local deployment
  ☆112Updated 3 months ago
• intel / intel-xpu-backend-for-triton
  OpenAI Triton backend for Intel® GPUs
  ☆193Updated this week
• OpenBMB / CPM.cu
  CPM.cu is a lightweight, high-performance CUDA implementation for LLMs, optimized for end-device inference and featuring cutting-edge tec…
  ☆160Updated this week
• huggingface / optimum-amd
  AMD related optimizations for transformer models
  ☆80Updated 3 weeks ago
• vllm-project / flash-attention
  Fast and memory-efficient exact attention
  ☆80Updated last week
• intel / torch-xpu-ops
  ☆48Updated this week
• intel / intel-extension-for-deepspeed
  Intel® Extension for DeepSpeed* is an extension to DeepSpeed that brings feature support with SYCL kernels on Intel GPU(XPU) device. Note…
  ☆61Updated 2 weeks ago
• bytedance / ByteMLPerf
  AI Accelerator Benchmark focuses on evaluating AI Accelerators from a practical production perspective, including the ease of use and ver…
  ☆251Updated 2 weeks ago
• sihyeong / Awesome-LLM-Inference-Engine
  ☆84Updated last month
• zartbot / shallowsim
  DeepSeek-V3/R1 inference performance simulator
  ☆155Updated 3 months ago
• intel / npu-nn-cost-model
  Library for modelling performance costs of different Neural Network workloads on NPU devices
  ☆34Updated last month
• microsoft / ark
  A GPU-driven system framework for scalable AI applications
  ☆117Updated 5 months ago
• mlc-ai / relax
  ☆161Updated last week
• intel / xFasterTransformer
  ☆428Updated last week
• Terapines / AI-Benchmark
  RISCV C and Triton AI-Benchmark
  ☆19Updated 8 months ago
• pytorch-labs / triton-cpu
  An experimental CPU backend for Triton (https//github.com/openai/triton)
  ☆43Updated 4 months ago
• AutonomicPerfectionist / PipeInfer
  PipeInfer: Accelerating LLM Inference using Asynchronous Pipelined Speculation
  ☆30Updated 8 months ago
• infinigence / SpecEE
  Repo for SpecEE: Accelerating Large Language Model Inference with Speculative Early Exiting (ISCA25)
  ☆45Updated 2 months ago
• FlagOpen / FlagCX
  ☆80Updated this week
• HabanaAI / vllm-fork
  A high-throughput and memory-efficient inference and serving engine for LLMs
  ☆77Updated this week
• BBuf / tensorrt-llm-moe
  ☆31Updated 5 months ago
• flexflow / flexflow-serve
  FlexFlow Serve: Low-Latency, High-Performance LLM Serving
  ☆48Updated 2 months ago
• opea-project / docs
  This repo contains documents of the OPEA project
  ☆42Updated last week