Alternatives and similar repositories for onnxruntime-server

Users that are interested in onnxruntime-server are comparing it to the libraries listed below

• rtzr / tritony
  Tiny configuration for Triton Inference Server
  ☆45Updated last year
• kibae / pg_onnx
  pg_onnx: ONNX Runtime integrated with PostgreSQL. Perform ML inference with data in your database.
  ☆57Updated 3 months ago
• RahulSChand / llama2.c-for-dummies
  Step by step explanation/tutorial of llama2.c
  ☆226Updated 2 years ago
• sionic-ai / BGE-M3-Model-Converter
  A tool for manual conversion of BGE-M3 models with preserved trainable variables and direct control over model outputs.
  ☆44Updated 5 months ago
• jwj7140 / Gugugo
  Gugugo: 한국어 오픈소스 번역 모델 프로젝트
  ☆84Updated last year
• dnotitia / smoothie-qwen
  A lightweight adjustment tool for smoothing token probabilities in the Qwen models to encourage balanced multilingual generation.
  ☆104Updated 6 months ago
• 42dot / 42dot_LLM
  42dot LLM consists of a pre-trained language model, 42dot LLM-PLM, and a fine-tuned model, 42dot LLM-SFT, which is trained to respond to …
  ☆130Updated last year
• Curt-Park / mnist-fastapi-celery-triton
  Simple example of FastAPI + Celery + Triton for benchmarking
  ☆64Updated 3 years ago
• mirusu400 / CLOVA-X
  Unofficial API for CLOVA X
  ☆37Updated 2 years ago
• lablup / all-smi
  Command-line utility for monitoring GPU hardware.
  ☆113Updated this week
• SqueezeBits / Torch-TRTLLM
  Ditto is an open-source framework that enables direct conversion of HuggingFace PreTrainedModels into TensorRT-LLM engines.
  ☆55Updated 6 months ago
• sionic-ai / Llama4-Token-Editor
  ☆64Updated 6 months ago
• Curt-Park / mnist-fastapi-aio-triton
  Simple example of FastAPI + gRPC AsyncIO + Triton
  ☆69Updated 3 years ago
• Marker-Inc-Korea / RAGchain
  Extension of Langchain for RAG. Easy benchmarking, multiple retrievals, reranker, time-aware RAG, and so on...
  ☆284Updated 2 years ago
• Curt-Park / triton-inference-server-practice
  Archives for Triton Inference Server Practices
  ☆15Updated 3 years ago
• wangkuiyi / huggingface-tokenizer-in-cxx
  ☆70Updated 2 years ago
• ainbr / chatgpt-weak-labeler-webui
  Weak Labeling (NER) using ChatGPT
  ☆36Updated 2 years ago
• Marker-Inc-Korea / Korean-SAT-LLM-Leaderboard
  Korean SAT leader board
  ☆168Updated 2 months ago
• nnstreamer / nnstreamer-example
  Example applications of nnstreamer. Note that we have to enable the 'apptest" CI module in the near future.
  ☆83Updated 2 months ago
• triton-inference-server / tensorrt_backend
  The Triton backend for TensorRT.
  ☆85Updated this week
• gyunggyung / KoAlpaca.cpp
  Locally run an Instruction-Tuned Chat-Style LLM KoAlpaca
  ☆24Updated 2 years ago
• LG-AI-EXAONE / EXAONE-3.5
  Official repository for EXAONE 3.5 built by LG AI Research
  ☆201Updated last year
• StableFluffy / EasyLLMFeaturePorter
  1-Click is all you need.
  ☆63Updated last year
• qwopqwop200 / GPTQ-for-KoAlpaca
  ☆15Updated 2 years ago
• instructkr / LogicKor
  한국어 언어모델 다분야 사고력 벤치마크
  ☆201Updated last year
• developer0hye / onnxruntime-cuda-cpp-example
  Examples for inference models with ONNXRuntime and CUDA
  ☆26Updated 2 years ago
• inisis / OnnxSlim
  A Toolkit to Help Optimize Onnx Model
  ☆409Updated this week
• annotation-ai / python-project-template
  Python Project Template
  ☆68Updated 3 years ago
• sionic-ai / xionic
  ☆48Updated last year
• triton-inference-server / common
  Common source, scripts and utilities shared across all Triton repositories.
  ☆79Updated 3 weeks ago