sunhailin-Leo / triton-service-goLinks
Unofficial golang package for the Triton Inference Server(https://github.com/triton-inference-server/server)
☆50Updated this week
Alternatives and similar repositories for triton-service-go
Users that are interested in triton-service-go are comparing it to the libraries listed below
Sorting:
- Go bindings for Tiktoken & HuggingFace Tokenizer☆172Updated 2 weeks ago
- A Go idiomatic binding to the C++ core of PyTorch☆350Updated 2 years ago
- LibTorch (PyTorch) bindings for Golang☆271Updated 4 years ago
- Go binding for Pytorch C++ API (libtorch)☆627Updated 7 months ago
- Common source, scripts and utilities shared across all Triton repositories.☆76Updated 2 weeks ago
- NLP tokenizers written in Go language☆272Updated last week
- The Triton backend for the ONNX Runtime.☆162Updated 2 weeks ago
- Go bindings for Faiss☆95Updated last year
- A Go (golang) library wrapping microsoft/onnxruntime.☆383Updated 2 months ago
- Go Bindings for the NVIDIA Management Library (NVML)☆401Updated 2 weeks ago
- Common source, scripts and utilities for creating Triton backends.☆347Updated 2 weeks ago
- Binding to transformers in ggml☆63Updated 3 weeks ago
- NLP transformers written in Go☆241Updated 2 years ago
- ☆296Updated last week
- The core library and APIs implementing the Triton Inference Server.☆150Updated last week
- Go framework for DL model inference and API deployment☆50Updated 9 months ago
- Golang bindings for Nvidia Datacenter GPU Manager (DCGM)☆133Updated last month
- onnx-go gives the ability to import a pre-trained neural network within Go without being linked to a framework or library.☆830Updated last year
- Triton Model Navigator is an inference toolkit designed for optimizing and deploying Deep Learning models with a focus on NVIDIA GPUs.☆211Updated 5 months ago
- Go binding to TensorRT C API to do inference with pre-trained model in Go☆30Updated 5 years ago
- Inference Llama 2 in Go☆39Updated 2 years ago
- Triton Inference Server Web UI☆15Updated last year
- Go Bindings for BERT NLP Models☆106Updated 6 years ago
- Triton Model Analyzer is a CLI tool to help with better understanding of the compute and memory requirements of the Triton Inference Serv…☆490Updated 2 weeks ago
- LLaMA-2 in native Go☆194Updated 9 months ago
- Triton backend for managing the model state tensors automatically in sequence batcher☆18Updated last year
- Triton Python, C++ and Java client libraries, and GRPC-generated client examples for go, java and scala.☆647Updated 2 weeks ago
- ☆412Updated last year
- OpenAI compatible API for TensorRT LLM triton backend☆215Updated last year
- Module, Model, and Tensor Serialization/Deserialization☆267Updated last month