huggingface / tgi-gaudiLinks
Large Language Model Text Generation Inference on Habana Gaudi
โ34Updated 7 months ago
Alternatives and similar repositories for tgi-gaudi
Users that are interested in tgi-gaudi are comparing it to the libraries listed below
Sorting:
- Easy and lightning fast training of ๐ค Transformers on Habana Gaudi processor (HPU)โ199Updated last week
- A high-throughput and memory-efficient inference and serving engine for LLMsโ83Updated this week
- ๐๏ธ A unified multi-backend utility for benchmarking Transformers, Timm, PEFT, Diffusers and Sentence-Transformers with full support of Oโฆโ318Updated last month
- A high-throughput and memory-efficient inference and serving engine for LLMsโ266Updated last year
- Benchmark suite for LLMs from Fireworks.aiโ83Updated 2 weeks ago
- ArcticInference: vLLM plugin for high-throughput, low-latency inferenceโ288Updated this week
- A unified library for building, evaluating, and storing speculative decoding algorithms for LLM inference in vLLMโ60Updated last week
- Easy and Efficient Quantization for Transformersโ202Updated 4 months ago
- Pretrain, finetune and serve LLMs on Intel platforms with Rayโ132Updated last month
- Intelยฎ Extension for DeepSpeed* is an extension to DeepSpeed that brings feature support with SYCL kernels on Intel GPU(XPU) device. Noteโฆโ63Updated 4 months ago
- โ258Updated last week
- โ302Updated this week
- โ218Updated 9 months ago
- IBM development fork of https://github.com/huggingface/text-generation-inferenceโ61Updated last month
- vLLM: A high-throughput and memory-efficient inference and serving engine for LLMsโ90Updated this week
- โ115Updated 2 weeks ago
- Reference models for Intel(R) Gaudi(R) AI Acceleratorโ165Updated last month
- โ205Updated 5 months ago
- KV cache compression for high-throughput LLM inferenceโ143Updated 8 months ago
- Genai-bench is a powerful benchmark tool designed for comprehensive token-level performance evaluation of large language model (LLM) servโฆโ222Updated this week
- JetStream is a throughput and memory optimized engine for LLM inference on XLA devices, starting with TPUs (and GPUs in future -- PRs welโฆโ385Updated 4 months ago
- ๐ Collection of components for development, training, tuning, and inference of foundation models leveraging PyTorch native components.โ215Updated last week
- vLLM performance dashboardโ37Updated last year
- โ56Updated 11 months ago
- Common recipes to run vLLMโ172Updated last week
- ๐ค Optimum Intel: Accelerate inference with Intel optimization toolsโ502Updated this week
- โ58Updated last year
- QUICK: Quantization-aware Interleaving and Conflict-free Kernel for efficient LLM inferenceโ118Updated last year
- A safetensors extension to efficiently store sparse quantized tensors on diskโ180Updated last week
- torchcomms: a modern PyTorch communications APIโ103Updated this week