microsoft / onnxruntime-genaiLinks
Generative AI extensions for onnxruntime
☆861Updated this week
Alternatives and similar repositories for onnxruntime-genai
Users that are interested in onnxruntime-genai are comparing it to the libraries listed below
Sorting:
- Olive: Simplify ML Model Finetuning, Conversion, Quantization, and Optimization for CPUs, GPUs and NPUs.☆2,159Updated this week
- onnxruntime-extensions: A specialized pre- and post- processing library for ONNX Runtime☆418Updated this week
- Run Generative AI models with simple C++/Python API and using OpenVINO Runtime☆364Updated this week
- Examples for using ONNX Runtime for model training.☆351Updated last year
- 🤗 Optimum Intel: Accelerate inference with Intel optimization tools☆502Updated this week
- ONNX Script enables developers to naturally author ONNX functions and models using a subset of Python.☆404Updated this week
- The Qualcomm® AI Hub Models are a collection of state-of-the-art machine learning models optimized for performance (latency, memory etc.)…☆808Updated last week
- An innovative library for efficient LLM inference via low-bit quantization☆349Updated last year
- Low-bit LLM inference on CPU/NPU with lookup table☆876Updated 4 months ago
- LiteRT continues the legacy of TensorFlow Lite as the trusted, high-performance runtime for on-device AI. Now with LiteRT Next, we're exp…☆872Updated last week
- Intel® NPU Acceleration Library☆692Updated 6 months ago
- No-code CLI designed for accelerating ONNX workflows☆215Updated 4 months ago
- ☆1,027Updated last year
- LLM model quantization (compression) toolkit with hw acceleration support for Nvidia CUDA, AMD ROCm, Intel XPU and Intel/AMD/Apple CPU vi…☆842Updated this week
- Advanced Quantization Algorithm for LLMs and VLMs, with support for CPU, Intel GPU, CUDA and HPU.☆668Updated last week
- Common utilities for ONNX converters☆281Updated last month
- The Triton TensorRT-LLM Backend☆903Updated this week
- Universal cross-platform tokenizers binding to HF and sentencepiece☆400Updated 2 months ago
- A unified library of state-of-the-art model optimization techniques like quantization, pruning, distillation, speculative decoding, etc. …☆1,464Updated this week
- Pure C++ implementation of several models for real-time chatting on your computer (CPU & GPU)☆721Updated last week
- A Python package for extending the official PyTorch that can easily obtain performance on Intel platform☆1,977Updated last week
- ⚡ Build your chatbot within minutes on your favorite device; offer SOTA compression techniques for LLMs; run LLMs efficiently on Intel Pl…☆2,164Updated last year
- Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM☆2,106Updated last week
- A pytorch quantization backend for optimum☆999Updated this week
- Use safetensors with ONNX 🤗☆73Updated 3 weeks ago
- Official implementation of Half-Quadratic Quantization (HQQ)☆884Updated this week
- TinyChatEngine: On-Device LLM Inference Library☆906Updated last year
- ☆430Updated last week
- SOTA low-bit LLM quantization (INT8/FP8/INT4/FP4/NF4) & sparsity; leading model compression techniques on TensorFlow, PyTorch, and ONNX R…☆2,511Updated last week
- The Qualcomm® AI Hub apps are a collection of state-of-the-art machine learning models optimized for performance (latency, memory etc.) a…☆320Updated last week