Alternatives and similar repositories for stateful_backend:

Users that are interested in stateful_backend are comparing it to the libraries listed below

• triton-inference-server / onnxruntime_backend
  The Triton backend for the ONNX Runtime.
  ☆140Updated 2 weeks ago
• triton-inference-server / redis_cache
  TRITONCACHE implementation of a Redis cache
  ☆13Updated 2 weeks ago
• triton-inference-server / tensorrt_backend
  The Triton backend for TensorRT.
  ☆70Updated 2 weeks ago
• wangkuiyi / huggingface-tokenizer-in-cxx
  ☆60Updated 2 years ago
• ray-project / ray_shuffling_data_loader
  A Ray-based data loader with per-epoch shuffling and configurable pipelining, for shuffling and loading training data for distributed tra…
  ☆18Updated 2 years ago
• Snowflake-Labs / vllm
  ☆14Updated last month
• triton-inference-server / triton_cli
  Triton CLI is an open source command line interface that enables users to create, deploy, and profile models served by the Triton Inferen…
  ☆61Updated last week
• ray-project / distml
  Distributed ML Optimizer
  ☆30Updated 3 years ago
• triton-inference-server / openvino_backend
  OpenVINO backend for Triton.
  ☆31Updated 2 weeks ago
• ray-project / mlflow-ray-serve
  MLFlow Deployment Plugin for Ray Serve
  ☆44Updated 2 years ago
• triton-inference-server / common
  Common source, scripts and utilities shared across all Triton repositories.
  ☆69Updated this week
• triton-inference-server / model_navigator
  Triton Model Navigator is an inference toolkit designed for optimizing and deploying Deep Learning models with a focus on NVIDIA GPUs.
  ☆199Updated 2 months ago
• sdpython / onnxcustom
  Tutorial on how to convert machine learned models into ONNX
  ☆16Updated 2 years ago
• NVIDIA / LDDL
  Distributed preprocessing and data loading for language datasets
  ☆39Updated 11 months ago
• triton-inference-server / pytorch_backend
  The Triton backend for the PyTorch TorchScript models.
  ☆144Updated 2 weeks ago
• tensorchord / inference-benchmark
  Benchmark for machine learning model online serving (LLM, embedding, Stable-Diffusion, Whisper)
  ☆28Updated last year
• AniZpZ / smoothquant
  [ICML 2023] SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Models
  ☆11Updated last year
• triton-inference-server / core
  The core library and APIs implementing the Triton Inference Server.
  ☆123Updated last week
• zhisbug / ray-scalable-ml-design
  Some microbenchmarks and design docs before commencement
  ☆12Updated 4 years ago
• qdrant / quaterion-models
  The collection of bulding blocks building fine-tunable metric learning models
  ☆32Updated 2 months ago
• fpaupier / gRPC-multiprocessing
  A boilerplate to use multiprocessing for your gRPC server in your Python project
  ☆25Updated 3 years ago
• UmerHA / triton_util
  Make triton easier
  ☆47Updated 9 months ago
• opendatahub-io / vllm-tgis-adapter
  vLLM adapter for a TGIS-compatible gRPC server.
  ☆25Updated this week
• facebookresearch / FAMBench
  Benchmarks to capture important workloads.
  ☆30Updated 2 months ago
• nod-ai / transformer-benchmarks
  benchmarking some transformer deployments
  ☆26Updated 2 years ago
• Systemcluster / kitoken
  Fast and versatile tokenizer for language models, compatible with SentencePiece, Tokenizers, Tiktoken and more. Supports BPE, Unigram and…
  ☆19Updated last week
• facebookresearch / NasRec
  NASRec Weight Sharing Neural Architecture Search for Recommender Systems
  ☆29Updated last year
• withmartian / leaderboard-backend
  Open sourced backend for Martian's LLM Inference Provider Leaderboard
  ☆17Updated 7 months ago
• lianakoleva / no-libtorch-compile
  ☆21Updated 3 weeks ago
• nvidia-riva / riva-asrlib-decoder
  Standalone implementation of the CUDA-accelerated WFST Decoder available in Riva
  ☆85Updated last month