huggingface / tgi-gaudiLinks
Large Language Model Text Generation Inference on Habana Gaudi
โ34Updated 4 months ago
Alternatives and similar repositories for tgi-gaudi
Users that are interested in tgi-gaudi are comparing it to the libraries listed below
Sorting:
- Easy and lightning fast training of ๐ค Transformers on Habana Gaudi processor (HPU)โ191Updated this week
- A high-throughput and memory-efficient inference and serving engine for LLMsโ78Updated this week
- Intelยฎ Extension for DeepSpeed* is an extension to DeepSpeed that brings feature support with SYCL kernels on Intel GPU(XPU) device. Noteโฆโ61Updated last month
- ๐๏ธ A unified multi-backend utility for benchmarking Transformers, Timm, PEFT, Diffusers and Sentence-Transformers with full support of Oโฆโ307Updated 2 months ago
- Benchmark suite for LLMs from Fireworks.aiโ76Updated this week
- Pretrain, finetune and serve LLMs on Intel platforms with Rayโ128Updated 3 weeks ago
- A high-throughput and memory-efficient inference and serving engine for LLMsโ266Updated 9 months ago
- Reference models for Intel(R) Gaudi(R) AI Acceleratorโ167Updated last week
- โ58Updated 10 months ago
- โ280Updated last week
- โ55Updated 8 months ago
- Easy and Efficient Quantization for Transformersโ198Updated last month
- โ98Updated this week
- vLLM performance dashboardโ33Updated last year
- ๐ค Optimum Intel: Accelerate inference with Intel optimization toolsโ481Updated this week
- Fast and memory-efficient exact attentionโ83Updated this week
- Dynamic batching library for Deep Learning inference. Tutorials for LLM, GPT scenarios.โ101Updated 11 months ago
- โ232Updated this week
- JetStream is a throughput and memory optimized engine for LLM inference on XLA devices, starting with TPUs (and GPUs in future -- PRs welโฆโ364Updated last month
- vLLM: A high-throughput and memory-efficient inference and serving engine for LLMsโ87Updated this week
- Module, Model, and Tensor Serialization/Deserializationโ250Updated this week
- A tool to configure, launch and manage your machine learning experiments.โ174Updated last week
- โ41Updated this week
- An innovative library for efficient LLM inference via low-bit quantizationโ349Updated 11 months ago
- ArcticInference: vLLM plugin for high-throughput, low-latency inferenceโ203Updated this week
- The Triton backend for the ONNX Runtime.โ156Updated this week
- Perplexity GPU Kernelsโ418Updated 2 weeks ago
- IBM development fork of https://github.com/huggingface/text-generation-inferenceโ61Updated 2 months ago
- โ195Updated 3 months ago
- โ215Updated 6 months ago