deepjavalibrary / djl-servingLinks
A universal scalable machine learning model deployment solution
☆238Updated last week
Alternatives and similar repositories for djl-serving
Users that are interested in djl-serving are comparing it to the libraries listed below
Sorting:
- ☆110Updated 9 months ago
- Example code for AWS Neuron SDK developers building inference and training applications☆149Updated 2 weeks ago
- Easy, fast and very cheap training and inference on AWS Trainium and Inferentia chips.☆245Updated this week
- ☆302Updated this week
- Large Language Model Hosting Container☆90Updated 2 weeks ago
- ☆60Updated last month
- Triton backend that enables pre-process, post-processing and other logic to be implemented in Python.☆651Updated 2 weeks ago
- Common source, scripts and utilities for creating Triton backends.☆352Updated 2 weeks ago
- Triton Model Analyzer is a CLI tool to help with better understanding of the compute and memory requirements of the Triton Inference Serv…☆495Updated this week
- Examples on how to use LangChain and Ray☆229Updated 2 years ago
- LLMPerf is a library for validating and benchmarking LLMs☆1,032Updated 10 months ago
- Triton Model Navigator is an inference toolkit designed for optimizing and deploying Deep Learning models with a focus on NVIDIA GPUs.☆213Updated 6 months ago
- Foundation model benchmarking tool. Run any model on any AWS platform and benchmark for performance across instance type and serving stac…☆254Updated 6 months ago
- ☆268Updated 6 months ago
- Toolkit for allowing inference and serving with PyTorch on SageMaker. Dockerfiles used for building SageMaker Pytorch Containers are at h…☆140Updated last year
- The Triton TensorRT-LLM Backend☆903Updated this week
- Powering AWS purpose-built machine learning chips. Blazing fast and cost effective, natively integrated into PyTorch and TensorFlow and i…☆548Updated last week
- Hands-on workshop for distributed training and hosting on SageMaker☆148Updated last week
- This repository contains tutorials and examples for Triton Inference Server☆792Updated 2 weeks ago
- Foundation Model Evaluations Library☆265Updated 2 months ago
- A high-throughput and memory-efficient inference and serving engine for LLMs☆21Updated last week
- 🏋️ A unified multi-backend utility for benchmarking Transformers, Timm, PEFT, Diffusers and Sentence-Transformers with full support of O…☆318Updated last month
- 🆕 Find the k-nearest neighbors (k-NN) for your vector data☆201Updated this week
- Find the optimal model serving solution for 🤗 Hugging Face models 🚀☆44Updated 3 months ago
- This is suite of the hands-on training materials that shows how to scale CV, NLP, time-series forecasting workloads with Ray.☆435Updated last year
- Pretrain, finetune and serve LLMs on Intel platforms with Ray☆132Updated last month
- Triton Python, C++ and Java client libraries, and GRPC-generated client examples for go, java and scala.☆654Updated last week
- A high-performance inference system for large language models, designed for production environments.☆479Updated 3 weeks ago
- PyTriton is a Flask/FastAPI-like interface that simplifies Triton's deployment in Python environments.☆823Updated 2 months ago
- The Triton backend for the ONNX Runtime.☆162Updated 2 weeks ago