huggingface / tgi-gaudiLinks
Large Language Model Text Generation Inference on Habana Gaudi
☆34Updated 6 months ago
Alternatives and similar repositories for tgi-gaudi
Users that are interested in tgi-gaudi are comparing it to the libraries listed below
Sorting:
- Easy and lightning fast training of 🤗 Transformers on Habana Gaudi processor (HPU)☆198Updated this week
- A high-throughput and memory-efficient inference and serving engine for LLMs☆83Updated last week
- Intel® Extension for DeepSpeed* is an extension to DeepSpeed that brings feature support with SYCL kernels on Intel GPU(XPU) device. Note…☆63Updated 3 months ago
- Reference models for Intel(R) Gaudi(R) AI Accelerator☆165Updated 2 weeks ago
- 🏋️ A unified multi-backend utility for benchmarking Transformers, Timm, PEFT, Diffusers and Sentence-Transformers with full support of O…☆315Updated 2 weeks ago
- Benchmark suite for LLMs from Fireworks.ai☆83Updated this week
- A high-throughput and memory-efficient inference and serving engine for LLMs☆265Updated 11 months ago
- Easy and Efficient Quantization for Transformers☆203Updated 3 months ago
- Pretrain, finetune and serve LLMs on Intel platforms with Ray☆132Updated 2 weeks ago
- IBM development fork of https://github.com/huggingface/text-generation-inference☆61Updated 3 weeks ago
- ☆255Updated last week
- Google TPU optimizations for transformers models☆120Updated 8 months ago
- ArcticInference: vLLM plugin for high-throughput, low-latency inference☆270Updated this week
- ☆218Updated 8 months ago
- ☆72Updated 6 months ago
- vLLM: A high-throughput and memory-efficient inference and serving engine for LLMs☆90Updated this week
- JetStream is a throughput and memory optimized engine for LLM inference on XLA devices, starting with TPUs (and GPUs in future -- PRs wel…☆379Updated 3 months ago
- A unified library for building, evaluating, and storing speculative decoding algorithms for LLM inference in vLLM☆53Updated this week
- A tool to configure, launch and manage your machine learning experiments.☆195Updated last week
- 🚀 Collection of components for development, training, tuning, and inference of foundation models leveraging PyTorch native components.☆213Updated last week
- ☆56Updated 10 months ago
- A safetensors extension to efficiently store sparse quantized tensors on disk☆167Updated this week
- PyTorch/XLA integration with JetStream (https://github.com/google/JetStream) for LLM inference"☆74Updated 3 weeks ago
- Module, Model, and Tensor Serialization/Deserialization☆267Updated last month
- Explore training for quantized models☆24Updated 2 months ago
- An innovative library for efficient LLM inference via low-bit quantization☆350Updated last year
- ☆58Updated last year
- ☆298Updated last week
- Fast low-bit matmul kernels in Triton☆376Updated last week
- Boosting 4-bit inference kernels with 2:4 Sparsity☆82Updated last year