huggingface / tgi-gaudiLinks
Large Language Model Text Generation Inference on Habana Gaudi
โ33Updated 3 months ago
Alternatives and similar repositories for tgi-gaudi
Users that are interested in tgi-gaudi are comparing it to the libraries listed below
Sorting:
- A high-throughput and memory-efficient inference and serving engine for LLMsโ76Updated this week
- Easy and lightning fast training of ๐ค Transformers on Habana Gaudi processor (HPU)โ188Updated this week
- Pretrain, finetune and serve LLMs on Intel platforms with Rayโ129Updated last month
- DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.โ13Updated last month
- Benchmark suite for LLMs from Fireworks.aiโ76Updated 2 weeks ago
- Intelยฎ Extension for DeepSpeed* is an extension to DeepSpeed that brings feature support with SYCL kernels on Intel GPU(XPU) device. Noteโฆโ61Updated 3 months ago
- โ55Updated 9 months ago
- IBM development fork of https://github.com/huggingface/text-generation-inferenceโ60Updated last month
- Reference models for Intel(R) Gaudi(R) AI Acceleratorโ163Updated last month
- โ159Updated this week
- vLLM performance dashboardโ30Updated last year
- ๐ Collection of components for development, training, tuning, and inference of foundation models leveraging PyTorch native components.โ204Updated this week
- Ongoing research training transformer models at scaleโ23Updated last week
- Easy and Efficient Quantization for Transformersโ199Updated 4 months ago
- KV cache compression for high-throughput LLM inferenceโ131Updated 4 months ago
- ๐๏ธ A unified multi-backend utility for benchmarking Transformers, Timm, PEFT, Diffusers and Sentence-Transformers with full support of Oโฆโ304Updated 3 weeks ago
- [ICLR2025] Breaking Throughput-Latency Trade-off for Long Sequences with Speculative Decodingโ116Updated 6 months ago
- QUICK: Quantization-aware Interleaving and Conflict-free Kernel for efficient LLM inferenceโ118Updated last year
- โ72Updated 2 months ago
- โ38Updated this week
- โ213Updated 5 months ago
- PyTorch/XLA integration with JetStream (https://github.com/google/JetStream) for LLM inference"โ62Updated 2 months ago
- โ46Updated this week
- โ54Updated 7 months ago
- A high-throughput and memory-efficient inference and serving engine for LLMsโ264Updated 8 months ago
- oneCCL Bindings for Pytorch*โ97Updated 2 months ago
- A tool to configure, launch and manage your machine learning experiments.โ161Updated this week
- โ119Updated last year
- Example ML projects that use the Determined library.โ32Updated 9 months ago
- โ19Updated this week