Alternatives and similar repositories for zero-copy-model-loading:

Users that are interested in zero-copy-model-loading are comparing it to the libraries listed below

• bentoml / simple_di
  Simple dependency injection framework for Python
  ☆20Updated 11 months ago
• ray-project / pygloo
  Pygloo provides Python bindings for Gloo.
  ☆22Updated last month
• hora-search / horapy
  🐍 Python bidding for the Hora Approximate Nearest Neighbor Search Algorithm library
  ☆72Updated 3 years ago
• mlflow / mlflow-torchserve
  Plugin for deploying MLflow models to TorchServe
  ☆108Updated 2 years ago
• triton-inference-server / pytorch_backend
  The Triton backend for the PyTorch TorchScript models.
  ☆146Updated this week
• coreweave / ml-containers
  ☆30Updated last week
• coreweave / tensorizer
  Module, Model, and Tensor Serialization/Deserialization
  ☆223Updated 2 months ago
• hamelsmu / llama-inference
  experiments with inference on llama
  ☆104Updated 10 months ago
• alvarobartt / safejax
  Serialize JAX, Flax, Haiku, or Objax model params with 🤗`safetensors`
  ☆44Updated 10 months ago
• allenai / cached_path
  A file utility for accessing both local and remote files through a unified interface.
  ☆40Updated last week
• pytorch-labs / torchfix
  TorchFix - a linter for PyTorch-using code with autofix support
  ☆138Updated 2 months ago
• bentoml / sentence-embedding-bento
  Sentence Embedding as a Service
  ☆15Updated last year
• Narsil / bloomserver
  ☆39Updated 2 years ago
• bentoml / IF-multi-GPUs-demo
  ☆12Updated last year
• google / jaxonnxruntime
  A user-friendly tool chain that enables the seamless execution of ONNX models using JAX as the backend.
  ☆109Updated last week
• Michaelvll / llm-ie-benchmarks
  A collection of reproducible inference engine benchmarks
  ☆24Updated this week
• pytorch / torchsnapshot
  A performant, memory-efficient checkpointing library for PyTorch applications, designed with large, complex distributed workloads in mind…
  ☆156Updated 4 months ago
• Yard1 / ray-skorch
  Distributed skorch on Ray Train
  ☆57Updated 2 years ago
• rwightman / genalog
  Genalog is an open source, cross-platform python package allowing generation of synthetic document images with custom degradations and te…
  ☆42Updated last year
• ray-project / mlflow-ray-serve
  MLFlow Deployment Plugin for Ray Serve
  ☆44Updated 3 years ago
• ezyang / torchdbg
  PyTorch centric eager mode debugger
  ☆47Updated 4 months ago
• fidelity / stoke
  A lightweight wrapper for PyTorch that provides a simple declarative API for context switching between devices, distributed modes, mixed-…
  ☆67Updated last year
• run-ai / rntop
  A top-like tool for monitoring GPUs in a cluster
  ☆86Updated last year
• octoml / octoml-profile
  Home for OctoML PyTorch Profiler
  ☆112Updated 2 years ago
• opendatahub-io / vllm-tgis-adapter
  vLLM adapter for a TGIS-compatible gRPC server.
  ☆26Updated this week
• triton-inference-server / onnxruntime_backend
  The Triton backend for the ONNX Runtime.
  ☆140Updated last week
• sdpython / mlprodict
  Productionize machine learning predictions, with ONNX or without
  ☆65Updated last year
• nod-ai / transformer-benchmarks
  benchmarking some transformer deployments
  ☆26Updated 2 years ago
• cortexlabs / nucleus
  Cortex-compatible model server for Python and TensorFlow
  ☆17Updated 2 years ago
• clearml / clearml-serving
  ClearML - Model-Serving Orchestration and Repository Solution
  ☆149Updated 3 months ago