Alternatives and similar repositories for zero-copy-model-loading

Users that are interested in zero-copy-model-loading are comparing it to the libraries listed below

• Michaelvll / llm-ie-benchmarks
  A collection of reproducible inference engine benchmarks
  ☆32Updated 2 months ago
• bentoml / simple_di
  Simple dependency injection framework for Python
  ☆21Updated last year
• foundation-model-stack / fastsafetensors
  High-performance safetensors model loader
  ☆48Updated 2 weeks ago
• graphcore-research / out-of-the-box-fp8-training
  Demo of the unit_scaling library, showing how a model can be easily adapted to train in FP8.
  ☆46Updated last year
• pytorch / torchsnapshot
  A performant, memory-efficient checkpointing library for PyTorch applications, designed with large, complex distributed workloads in mind…
  ☆158Updated 3 weeks ago
• hora-search / horapy
  🐍 Python bidding for the Hora Approximate Nearest Neighbor Search Algorithm library
  ☆72Updated 3 years ago
• nod-ai / SRT
  Nod.ai 🦈 version of 👻 . You probably want to start at https://github.com/nod-ai/shark for the product and the upstream IREE repository …
  ☆106Updated 6 months ago
• triton-inference-server / pytorch_backend
  The Triton backend for the PyTorch TorchScript models.
  ☆156Updated last week
• ray-project / pygloo
  Pygloo provides Python bindings for Gloo.
  ☆21Updated last week
• pytorch-labs / torchfix
  TorchFix - a linter for PyTorch-using code with autofix support
  ☆143Updated 5 months ago
• hamelsmu / llama-inference
  experiments with inference on llama
  ☆104Updated last year
• coreweave / tensorizer
  Module, Model, and Tensor Serialization/Deserialization
  ☆248Updated last week
• facebookresearch / PostText
  PostText is a QA system for querying your text data. When appropriate structured views are in place, PostText is good at answering querie…
  ☆32Updated 2 years ago
• onnx / neural-compressor
  Model compression for ONNX
  ☆96Updated 8 months ago
• gpuopenanalytics / pynvml
  Provide Python access to the NVML library for GPU diagnostics
  ☆241Updated 7 months ago
• triton-inference-server / onnxruntime_backend
  The Triton backend for the ONNX Runtime.
  ☆156Updated this week
• sdpython / mlprodict
  Productionize machine learning predictions, with ONNX or without
  ☆65Updated last year
• rwightman / genalog
  Genalog is an open source, cross-platform python package allowing generation of synthetic document images with custom degradations and te…
  ☆42Updated last year
• stas00 / ml-ways
  ML/DL Math and Method notes
  ☆61Updated last year
• pytorch / torchdistx
  Torch Distributed Experimental
  ☆116Updated 11 months ago
• RAIVNLab / AdANNS
  Code repository for the paper - "AdANNS: A Framework for Adaptive Semantic Search"
  ☆65Updated last year
• eth-easl / fmengine
  Utilities for Training Very Large Models
  ☆58Updated 9 months ago
• nicolargo / nvidia-ml-py3
  Python 3 Bindings for the NVIDIA Management Library
  ☆139Updated last year
• microsoft / batch-inference
  Dynamic batching library for Deep Learning inference. Tutorials for LLM, GPT scenarios.
  ☆99Updated 11 months ago
• nod-ai / transformer-benchmarks
  benchmarking some transformer deployments
  ☆26Updated 2 years ago
• JiahaoYao / awesome-ray
  Ray - A curated list of resources: https://github.com/ray-project/ray
  ☆65Updated 3 weeks ago
• ray-project / distml
  Distributed ML Optimizer
  ☆32Updated 3 years ago
• ryantd / veloce
  WIP. Veloce is a low-code Ray-based parallelization library that makes machine learning computation novel, efficient, and heterogeneous.
  ☆18Updated 2 years ago
• pytorch / multipy
  torch::deploy (multipy for non-torch uses) is a system that lets you get around the GIL problem by running multiple Python interpreters i…
  ☆180Updated last week
• hpcaitech / CachedEmbedding
  A memory efficient DLRM training solution using ColossalAI
  ☆105Updated 2 years ago