Alternatives and similar repositories for stateful_backend

Users that are interested in stateful_backend are comparing it to the libraries listed below

• triton-inference-server / onnxruntime_backend
  The Triton backend for the ONNX Runtime.
  ☆153Updated last week
• sdpython / onnxcustom
  Tutorial on how to convert machine learned models into ONNX
  ☆16Updated 2 years ago
• UmerHA / triton_util
  Make triton easier
  ☆46Updated last year
• chainyo / tensorshare
  🤝 Trade any tensors over the network
  ☆30Updated last year
• cortexlabs / nucleus
  Cortex-compatible model server for Python and TensorFlow
  ☆17Updated 2 years ago
• IlyasMoutawwakil / py-txi
  A Python wrapper around HuggingFace's TGI (text-generation-inference) and TEI (text-embedding-inference) servers.
  ☆33Updated last month
• triton-inference-server / triton_cli
  Triton CLI is an open source command line interface that enables users to create, deploy, and profile models served by the Triton Inferen…
  ☆64Updated 2 weeks ago
• BlackHC / llmtracer
  Trace LLM calls (and others) and visualize them in WandB, as interactive SVG or using a streaming local webapp
  ☆14Updated 4 months ago
• triton-inference-server / redis_cache
  TRITONCACHE implementation of a Redis cache
  ☆14Updated 2 weeks ago
• Knowledgator / FlashDeBERTa
  Trully flash implementation of DeBERTa disentangled attention mechanism.
  ☆58Updated last month
• triton-inference-server / tensorrt_backend
  The Triton backend for TensorRT.
  ☆77Updated last week
• iamlemec / bert.cpp
  GGML implementation of BERT model with Python bindings and quantization.
  ☆55Updated last year
• wangkuiyi / huggingface-tokenizer-in-cxx
  ☆67Updated 2 years ago
• nod-ai / transformer-benchmarks
  benchmarking some transformer deployments
  ☆26Updated 2 years ago
• Michaelvll / llm-ie-benchmarks
  A collection of reproducible inference engine benchmarks
  ☆31Updated 2 months ago
• Narsil / bloomserver
  ☆39Updated 2 years ago
• triton-inference-server / openvino_backend
  OpenVINO backend for Triton.
  ☆32Updated last week
• triton-inference-server / model_navigator
  Triton Model Navigator is an inference toolkit designed for optimizing and deploying Deep Learning models with a focus on NVIDIA GPUs.
  ☆205Updated 2 months ago
• qdrant / quaterion-models
  The collection of bulding blocks building fine-tunable metric learning models
  ☆32Updated 2 months ago
• NVIDIA / NeMo-speech-data-processor
  A toolkit for processing speech data and creating speech datasets
  ☆120Updated last week
• ray-project / ray_shuffling_data_loader
  A Ray-based data loader with per-epoch shuffling and configurable pipelining, for shuffling and loading training data for distributed tra…
  ☆18Updated 2 years ago
• triton-inference-server / common
  Common source, scripts and utilities shared across all Triton repositories.
  ☆74Updated last week
• mixedbread-ai / binary-embeddings
  Showcase how mxbai-embed-large-v1 can be used to produce binary embedding. Binary embeddings enabled 32x storage savings and 40x faster r…
  ☆18Updated last year
• triton-inference-server / pytorch_backend
  The Triton backend for the PyTorch TorchScript models.
  ☆152Updated last week
• flashlight / text
  Text utilities, including beam search decoding, tokenizing, and more, built for use in Flashlight.
  ☆70Updated 3 months ago
• JulesBelveze / bert-squeeze
  🛠️ Tools for Transformers compression using PyTorch Lightning ⚡
  ☆83Updated 7 months ago
• lessw2020 / transformer_central
  Various transformers for FSDP research
  ☆37Updated 2 years ago
• gnovack / distributed-training-and-deepspeed
  ☆17Updated 2 years ago
• ashvardanian / jaccard-index
  Optimizing bit-level Jaccard Index and Population Counts for large-scale quantized Vector Search via Harley-Seal CSA and Lookup Tables
  ☆20Updated last month
• speechmatics / ctranslate2_triton_backend
  Triton backend for https://github.com/OpenNMT/CTranslate2
  ☆35Updated last year