Alternatives and similar repositories for onnxruntime-server

Users that are interested in onnxruntime-server are comparing it to the libraries listed below

• kibae / pg_onnx
  pg_onnx: ONNX Runtime integrated with PostgreSQL. Perform ML inference with data in your database.
  ☆49Updated 3 months ago
• RahulSChand / llama2.c-for-dummies
  Step by step explanation/tutorial of llama2.c
  ☆223Updated last year
• rtzr / tritony
  Tiny configuration for Triton Inference Server
  ☆45Updated 7 months ago
• dnotitia / smoothie-qwen
  A lightweight adjustment tool for smoothing token probabilities in the Qwen models to encourage balanced multilingual generation.
  ☆80Updated last month
• Curt-Park / mnist-fastapi-aio-triton
  Simple example of FastAPI + gRPC AsyncIO + Triton
  ☆67Updated 3 years ago
• likejazz / llama3.cuda
  llama3.cuda is a pure C/CUDA implementation for Llama 3 model.
  ☆343Updated 4 months ago
• Curt-Park / mnist-fastapi-celery-triton
  Simple example of FastAPI + Celery + Triton for benchmarking
  ☆64Updated 3 years ago
• jwj7140 / Gugugo
  Gugugo: 한국어 오픈소스 번역 모델 프로젝트
  ☆81Updated last year
• sionic-ai / BGE-M3-Model-Converter
  A tool for manual conversion of BGE-M3 models with preserved trainable variables and direct control over model outputs.
  ☆42Updated 7 months ago
• wangkuiyi / huggingface-tokenizer-in-cxx
  ☆68Updated 2 years ago
• SqueezeBits / Torch-TRTLLM
  Ditto is an open-source framework that enables direct conversion of HuggingFace PreTrainedModels into TensorRT-LLM engines.
  ☆49Updated last month
• Marker-Inc-Korea / Korean-SAT-LLM-Leaderboard
  Korean SAT leader board
  ☆166Updated last week
• Marker-Inc-Korea / RAGchain
  Extension of Langchain for RAG. Easy benchmarking, multiple retrievals, reranker, time-aware RAG, and so on...
  ☆282Updated last year
• LG-AI-EXAONE / EXAONE-3.5
  Official repository for EXAONE 3.5 built by LG AI Research
  ☆200Updated 8 months ago
• triton-inference-server / tensorrt_backend
  The Triton backend for TensorRT.
  ☆77Updated last month
• mirusu400 / CLOVA-X
  Unofficial API for CLOVA X
  ☆37Updated last year
• inureyes / all-smi
  Command-line utility for monitoring GPU hardware.
  ☆82Updated last week
• sionic-ai / Llama4-Token-Editor
  ☆62Updated last month
• inisis / OnnxSlim
  A Toolkit to Help Optimize Onnx Model
  ☆205Updated 2 weeks ago
• qwopqwop200 / ko-arena-hard-auto
  Ko-Arena-Hard-Auto: An automatic LLM benchmark for Korean
  ☆23Updated 4 months ago
• triton-inference-server / onnxruntime_backend
  The Triton backend for the ONNX Runtime.
  ☆159Updated last week
• triton-inference-server / vllm_backend
  ☆293Updated last month
• LG-AI-EXAONE / EXAONE-Deep
  Official repository for EXAONE Deep built by LG AI Research
  ☆401Updated 3 months ago
• yvonwin / qwen2.cpp
  qwen2 and llama3 cpp implementation
  ☆47Updated last year
• kakao / varlog
  Scalable distributed log storage for strong consistency, total order, and high availability
  ☆51Updated last week
• rlaehdals / tfviz
  CLI tool to convert Terraform plan output into HTML visualizations
  ☆20Updated this week
• smarteasy / open-prompt
  ☆19Updated 4 months ago
• sionic-ai / serverless-rag-mcp-server
  ☆36Updated 5 months ago
• triton-inference-server / common
  Common source, scripts and utilities shared across all Triton repositories.
  ☆76Updated last month
• deep-diver / hf-daily-paper-newsletter
  Newsletter bot for 🤗 Daily Papers
  ☆127Updated this week