huggingface / optimum-intelLinks
π€ Optimum Intel: Accelerate inference with Intel optimization tools
β473Updated this week
Alternatives and similar repositories for optimum-intel
Users that are interested in optimum-intel are comparing it to the libraries listed below
Sorting:
- Easy and lightning fast training of π€ Transformers on Habana Gaudi processor (HPU)β188Updated this week
- Run Generative AI models with simple C++/Python API and using OpenVINO Runtimeβ293Updated this week
- An innovative library for efficient LLM inference via low-bit quantizationβ349Updated 9 months ago
- ποΈ A unified multi-backend utility for benchmarking Transformers, Timm, PEFT, Diffusers and Sentence-Transformers with full support of Oβ¦β304Updated 3 weeks ago
- SOTA low-bit LLM quantization (INT8/FP8/INT4/FP4/NF4) & sparsity; leading model compression techniques on TensorFlow, PyTorch, and ONNX Rβ¦β2,430Updated this week
- A Python package for extending the official PyTorch that can easily obtain performance on Intel platformβ1,875Updated last week
- Tools for easier OpenVINO development/debuggingβ9Updated 3 months ago
- ONNX Script enables developers to naturally author ONNX functions and models using a subset of Python.β358Updated this week
- OpenVINO Tokenizers extensionβ36Updated this week
- A curated list of OpenVINO based AI projectsβ137Updated last week
- Advanced Quantization Algorithm for LLMs and VLMs, with support for CPU, Intel GPU, CUDA and HPU. Seamlessly integrated with Torchao, Traβ¦β499Updated this week
- Triton Model Analyzer is a CLI tool to help with better understanding of the compute and memory requirements of the Triton Inference Servβ¦β477Updated last week
- β‘ Build your chatbot within minutes on your favorite device; offer SOTA compression techniques for LLMs; run LLMs efficiently on Intel Plβ¦β2,169Updated 8 months ago
- Examples for using ONNX Runtime for model training.β338Updated 7 months ago
- PyTriton is a Flask/FastAPI-like interface that simplifies Triton's deployment in Python environments.β801Updated 4 months ago
- A pytorch quantization backend for optimumβ955Updated this week
- A high-throughput and memory-efficient inference and serving engine for LLMsβ264Updated 8 months ago
- Generative AI extensions for onnxruntimeβ740Updated this week
- Common utilities for ONNX convertersβ272Updated 6 months ago
- onnxruntime-extensions: A specialized pre- and post- processing library for ONNX Runtimeβ393Updated this week
- The Triton backend for the ONNX Runtime.β152Updated this week
- Common source, scripts and utilities for creating Triton backends.β327Updated this week
- The Triton TensorRT-LLM Backendβ851Updated this week
- A unified library of state-of-the-art model optimization techniques like quantization, pruning, distillation, speculative decoding, etc. β¦β995Updated this week
- Reference models for Intel(R) Gaudi(R) AI Acceleratorβ163Updated last month
- FP16xINT4 LLM inference kernel that can achieve near-ideal ~4x speedups up to medium batchsizes of 16-32 tokens.β845Updated 9 months ago
- β20Updated 11 months ago
- β411Updated last year
- A general 2-8 bits quantization toolbox with GPTQ/AWQ/HQQ/VPTQ, and export to onnx/onnx-runtime easily.β172Updated 2 months ago
- β267Updated last week