Alternatives and similar repositories for tgi-gaudi

Users that are interested in tgi-gaudi are comparing it to the libraries listed below

• huggingface / optimum-habana
  Easy and lightning fast training of 🤗 Transformers on Habana Gaudi processor (HPU)
  ☆186Updated this week
• HabanaAI / vllm-fork
  A high-throughput and memory-efficient inference and serving engine for LLMs
  ☆70Updated this week
• intel / llm-on-ray
  Pretrain, finetune and serve LLMs on Intel platforms with Ray
  ☆126Updated 2 weeks ago
• HabanaAI / Model-References
  Reference models for Intel(R) Gaudi(R) AI Accelerator
  ☆162Updated 2 weeks ago
• intel / intel-extension-for-deepspeed
  Intel® Extension for DeepSpeed* is an extension to DeepSpeed that brings feature support with SYCL kernels on Intel GPU(XPU) device. Note…
  ☆61Updated 2 months ago
• huggingface / optimum-benchmark
  🏋️ A unified multi-backend utility for benchmarking Transformers, Timm, PEFT, Diffusers and Sentence-Transformers with full support of O…
  ☆300Updated this week
• NetEase-FuXi / EETQ
  Easy and Efficient Quantization for Transformers
  ☆197Updated 3 months ago
• neuralmagic / nm-vllm
  A high-throughput and memory-efficient inference and serving engine for LLMs
  ☆263Updated 7 months ago
• fw-ai / benchmark
  Benchmark suite for LLMs from Fireworks.ai
  ☆72Updated this week
• HabanaAI / DeepSpeed
  DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.
  ☆13Updated 2 months ago
• mlc-ai / llm-perf-bench
  ☆119Updated last year
• run-ai / llmperf
  ☆53Updated 7 months ago
• intel / torch-xpu-ops
  ☆45Updated this week
• opea-project / GenAIComps
  GenAI components at micro-service level; GenAI service composer to create mega-service
  ☆145Updated this week
• microsoft / sarathi-serve
  A low-latency & high-throughput serving engine for LLMs
  ☆354Updated 3 weeks ago
• wejoncy / QLLM
  A general 2-8 bits quantization toolbox with GPTQ/AWQ/HQQ/VPTQ, and export to onnx/onnx-runtime easily.
  ☆169Updated last month
• microsoft / vattention
  Dynamic Memory Management for Serving LLMs without PagedAttention
  ☆366Updated 3 weeks ago
• intel / torch-ccl
  oneCCL Bindings for Pytorch*
  ☆97Updated 2 weeks ago
• deepspeedai / DeepSpeed-Kernels
  ☆68Updated last month
• opea-project / docs
  This repo contains documents of the OPEA project
  ☆38Updated this week
• intel / intel-xpu-backend-for-triton
  OpenAI Triton backend for Intel® GPUs
  ☆184Updated this week
• triton-inference-server / vllm_backend
  ☆255Updated this week
• coreweave / nccl-tests
  NVIDIA NCCL Tests for Distributed Training
  ☆90Updated this week
• triton-inference-server / triton_distributed
  ☆49Updated 2 months ago
• openvinotoolkit / openvino_tokenizers
  OpenVINO Tokenizers extension
  ☆33Updated this week
• microsoft / mscclpp
  MSCCL++: A GPU-driven communication stack for scalable AI applications
  ☆349Updated this week
• coreweave / tensorizer
  Module, Model, and Tensor Serialization/Deserialization
  ☆227Updated this week
• ppl-ai / pplx-kernels
  Perplexity GPU Kernels
  ☆281Updated last week
• IsaacRe / vllm-kvcompress
  KV cache compression for high-throughput LLM inference
  ☆126Updated 3 months ago
• SqueezeBits / QUICK
  QUICK: Quantization-aware Interleaving and Conflict-free Kernel for efficient LLM inference
  ☆117Updated last year