kibae / onnxruntime-serverLinks
ONNX Runtime Server: The ONNX Runtime Server is a server that provides TCP and HTTP/HTTPS REST APIs for ONNX inference.
☆166Updated 2 months ago
Alternatives and similar repositories for onnxruntime-server
Users that are interested in onnxruntime-server are comparing it to the libraries listed below
Sorting:
- pg_onnx: ONNX Runtime integrated with PostgreSQL. Perform ML inference with data in your database.☆49Updated 2 months ago
- Tiny configuration for Triton Inference Server☆45Updated 7 months ago
- Step by step explanation/tutorial of llama2.c☆223Updated last year
- Simple example of FastAPI + Celery + Triton for benchmarking☆64Updated 3 years ago
- A tool for manual conversion of BGE-M3 models with preserved trainable variables and direct control over model outputs.☆42Updated 6 months ago
- Simple example of FastAPI + gRPC AsyncIO + Triton☆67Updated 2 years ago
- Gugugo: 한국어 오픈소스 번역 모델 프로젝트☆81Updated last year
- ☆62Updated 3 weeks ago
- A lightweight adjustment tool for smoothing token probabilities in the Qwen models to encourage balanced multilingual generation.☆78Updated last month
- Unofficial API for CLOVA X☆37Updated last year
- Extension of Langchain for RAG. Easy benchmarking, multiple retrievals, reranker, time-aware RAG, and so on...☆281Updated last year
- Example applications of nnstreamer. Note that we have to enable the 'apptest" CI module in the near future.☆81Updated 8 months ago
- Newsletter bot for 🤗 Daily Papers☆126Updated this week
- The Triton backend for TensorRT.☆77Updated this week
- ☆68Updated 2 years ago
- ☆36Updated 5 months ago
- Python Project Template☆68Updated 3 years ago
- 42dot LLM consists of a pre-trained language model, 42dot LLM-PLM, and a fine-tuned model, 42dot LLM-SFT, which is trained to respond to …☆130Updated last year
- A loader that lets you try running LLMs built for WebGPU.☆29Updated last year
- Ko-Arena-Hard-Auto: An automatic LLM benchmark for Korean☆23Updated 3 months ago
- ☆19Updated 3 months ago
- Building AI agent with hyperpocket tool in a flash☆50Updated 3 months ago
- qwen2 and llama3 cpp implementation☆46Updated last year
- Ditto is an open-source framework that enables direct conversion of HuggingFace PreTrainedModels into TensorRT-LLM engines.☆46Updated 3 weeks ago
- Korean SAT leader board☆167Updated 5 months ago
- The Triton backend for the ONNX Runtime.☆156Updated this week
- public repo for TANGO (Target Aware No-code neural network Generation and Operation framework)☆108Updated 2 months ago
- Official repository for EXAONE 3.5 built by LG AI Research☆200Updated 7 months ago
- A Toolkit to Help Optimize Onnx Model☆189Updated this week
- Command-line utility for monitoring GPU hardware.☆60Updated this week