huggingface / tgi-gaudiLinks
Large Language Model Text Generation Inference on Habana Gaudi
โ34Updated 5 months ago
Alternatives and similar repositories for tgi-gaudi
Users that are interested in tgi-gaudi are comparing it to the libraries listed below
Sorting:
- Easy and lightning fast training of ๐ค Transformers on Habana Gaudi processor (HPU)โ193Updated this week
- A high-throughput and memory-efficient inference and serving engine for LLMsโ80Updated this week
- A high-throughput and memory-efficient inference and serving engine for LLMsโ266Updated 10 months ago
- Benchmark suite for LLMs from Fireworks.aiโ80Updated 3 weeks ago
- Intelยฎ Extension for DeepSpeed* is an extension to DeepSpeed that brings feature support with SYCL kernels on Intel GPU(XPU) device. Noteโฆโ62Updated last month
- ๐๏ธ A unified multi-backend utility for benchmarking Transformers, Timm, PEFT, Diffusers and Sentence-Transformers with full support of Oโฆโ310Updated last week
- Pretrain, finetune and serve LLMs on Intel platforms with Rayโ128Updated 2 weeks ago
- Easy and Efficient Quantization for Transformersโ201Updated 2 months ago
- JetStream is a throughput and memory optimized engine for LLM inference on XLA devices, starting with TPUs (and GPUs in future -- PRs welโฆโ371Updated 2 months ago
- โ291Updated 3 weeks ago
- IBM development fork of https://github.com/huggingface/text-generation-inferenceโ61Updated 3 months ago
- Google TPU optimizations for transformers modelsโ118Updated 7 months ago
- Dynamic batching library for Deep Learning inference. Tutorials for LLM, GPT scenarios.โ103Updated last year
- โ34Updated last week
- โ55Updated 9 months ago
- ArcticInference: vLLM plugin for high-throughput, low-latency inferenceโ215Updated this week
- The Triton backend for the ONNX Runtime.โ159Updated 3 weeks ago
- A tool to configure, launch and manage your machine learning experiments.โ183Updated this week
- Module, Model, and Tensor Serialization/Deserializationโ258Updated last week
- vLLM adapter for a TGIS-compatible gRPC server.โ37Updated this week
- OpenAI compatible API for TensorRT LLM triton backendโ214Updated last year
- โ196Updated 3 months ago
- PyTorch/XLA integration with JetStream (https://github.com/google/JetStream) for LLM inference"โ68Updated 5 months ago
- Reference models for Intel(R) Gaudi(R) AI Acceleratorโ167Updated 3 weeks ago
- โ238Updated this week
- Inference server benchmarking toolโ93Updated 4 months ago
- ๐ค Optimum Intel: Accelerate inference with Intel optimization toolsโ485Updated this week
- A safetensors extension to efficiently store sparse quantized tensors on diskโ153Updated last week
- vLLM performance dashboardโ33Updated last year
- An innovative library for efficient LLM inference via low-bit quantizationโ349Updated 11 months ago