huggingface / tgi-gaudi
Large Language Model Text Generation Inference on Habana Gaudi
☆32Updated last month
Alternatives and similar repositories for tgi-gaudi:
Users that are interested in tgi-gaudi are comparing it to the libraries listed below
- A high-throughput and memory-efficient inference and serving engine for LLMs☆67Updated this week
- Easy and lightning fast training of 🤗 Transformers on Habana Gaudi processor (HPU)☆185Updated this week
- Intel® Extension for DeepSpeed* is an extension to DeepSpeed that brings feature support with SYCL kernels on Intel GPU(XPU) device. Note…☆62Updated last month
- Pretrain, finetune and serve LLMs on Intel platforms with Ray☆125Updated 3 weeks ago
- DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.☆13Updated last month
- 🏋️ A unified multi-backend utility for benchmarking Transformers, Timm, PEFT, Diffusers and Sentence-Transformers with full support of O…☆295Updated 2 months ago
- Reference models for Intel(R) Gaudi(R) AI Accelerator☆162Updated 2 weeks ago
- Benchmark suite for LLMs from Fireworks.ai☆70Updated 2 months ago
- GenAI components at micro-service level; GenAI service composer to create mega-service☆136Updated this week
- IBM development fork of https://github.com/huggingface/text-generation-inference☆60Updated 4 months ago
- Easy and Efficient Quantization for Transformers☆197Updated 2 months ago
- ☆68Updated 3 weeks ago
- ☆246Updated last week
- NVIDIA NCCL Tests for Distributed Training☆88Updated this week
- oneCCL Bindings for Pytorch*☆94Updated last week
- Google TPU optimizations for transformers models☆108Updated 3 months ago
- ☆49Updated 5 months ago
- A tool to configure, launch and manage your machine learning experiments.☆139Updated this week
- ArcticTraining is a framework designed to simplify and accelerate the post-training process for large language models (LLMs)☆66Updated this week
- ☆38Updated this week
- The Triton backend for the PyTorch TorchScript models.☆146Updated this week
- PyTorch/XLA integration with JetStream (https://github.com/google/JetStream) for LLM inference"☆59Updated 3 weeks ago
- A high-throughput and memory-efficient inference and serving engine for LLMs☆262Updated 6 months ago
- ☆196Updated 2 weeks ago
- ☆29Updated this week
- ☆53Updated 7 months ago
- OpenAI compatible API for TensorRT LLM triton backend☆205Updated 8 months ago
- Fast low-bit matmul kernels in Triton☆291Updated this week
- Evaluation, benchmark, and scorecard, targeting for performance on throughput and latency, accuracy on popular evaluation harness, safety…☆29Updated this week
- ☆61Updated this week