microsoft / onnxruntime-genaiLinks
Generative AI extensions for onnxruntime
☆825Updated this week
Alternatives and similar repositories for onnxruntime-genai
Users that are interested in onnxruntime-genai are comparing it to the libraries listed below
Sorting:
- Olive: Simplify ML Model Finetuning, Conversion, Quantization, and Optimization for CPUs, GPUs and NPUs.☆2,101Updated this week
- onnxruntime-extensions: A specialized pre- and post- processing library for ONNX Runtime☆411Updated this week
- 🤗 Optimum Intel: Accelerate inference with Intel optimization tools☆489Updated this week
- ONNX Script enables developers to naturally author ONNX functions and models using a subset of Python.☆381Updated this week
- Examples for using ONNX Runtime for model training.☆345Updated 10 months ago
- The Qualcomm® AI Hub Models are a collection of state-of-the-art machine learning models optimized for performance (latency, memory etc.)…☆782Updated 2 weeks ago
- Run Generative AI models with simple C++/Python API and using OpenVINO Runtime☆334Updated this week
- Low-bit LLM inference on CPU/NPU with lookup table☆852Updated 3 months ago
- An innovative library for efficient LLM inference via low-bit quantization☆348Updated last year
- Intel® NPU Acceleration Library☆689Updated 4 months ago
- ☆1,030Updated last year
- No-code CLI designed for accelerating ONNX workflows☆214Updated 3 months ago
- Common utilities for ONNX converters☆279Updated last week
- Advanced Quantization Algorithm for LLMs and VLMs, with support for CPU, Intel GPU, CUDA and HPU.☆614Updated this week
- The Triton TensorRT-LLM Backend☆887Updated this week
- LiteRT continues the legacy of TensorFlow Lite as the trusted, high-performance runtime for on-device AI. Now with LiteRT Next, we're exp…☆772Updated this week
- Official implementation of Half-Quadratic Quantization (HQQ)☆878Updated last week
- Universal cross-platform tokenizers binding to HF and sentencepiece☆388Updated last month
- A unified library of state-of-the-art model optimization techniques like quantization, pruning, distillation, speculative decoding, etc. …☆1,344Updated this week
- ⚡ Build your chatbot within minutes on your favorite device; offer SOTA compression techniques for LLMs; run LLMs efficiently on Intel Pl…☆2,170Updated 11 months ago
- Examples for using ONNX Runtime for machine learning inferencing.☆1,474Updated this week
- A pytorch quantization backend for optimum☆987Updated 3 weeks ago
- LLaMa/RWKV onnx models, quantization and testcase☆365Updated 2 years ago
- Inference Vision Transformer (ViT) in plain C/C++ with ggml☆294Updated last year
- Use safetensors with ONNX 🤗☆69Updated last week
- 🏋️ A unified multi-backend utility for benchmarking Transformers, Timm, PEFT, Diffusers and Sentence-Transformers with full support of O…☆315Updated this week
- A Python package for extending the official PyTorch that can easily obtain performance on Intel platform☆1,953Updated 2 weeks ago
- Supporting PyTorch models with the Google AI Edge TFLite runtime.☆777Updated this week
- Pure C++ implementation of several models for real-time chatting on your computer (CPU & GPU)☆696Updated this week
- LLM model quantization (compression) toolkit with hw acceleration support for Nvidia CUDA, AMD ROCm, Intel XPU and Intel/AMD/Apple CPU vi…☆778Updated this week