project-codeflare / zero-copy-model-loadingLinks
In-depth code associated with my Medium blog post, "How to Load PyTorch Models 340 Times Faster with Ray"
☆28Updated 3 years ago
Alternatives and similar repositories for zero-copy-model-loading
Users that are interested in zero-copy-model-loading are comparing it to the libraries listed below
Sorting:
- Simple dependency injection framework for Python☆21Updated last year
- Provide Python access to the NVML library for GPU diagnostics☆248Updated 3 weeks ago
- Module, Model, and Tensor Serialization/Deserialization☆268Updated last month
- High-performance safetensors model loader☆62Updated 2 months ago
- A collection of reproducible inference engine benchmarks☆33Updated 5 months ago
- The Triton backend for the PyTorch TorchScript models.☆159Updated last week
- WIP. Veloce is a low-code Ray-based parallelization library that makes machine learning computation novel, efficient, and heterogeneous.☆18Updated 3 years ago
- TorchFix - a linter for PyTorch-using code with autofix support☆149Updated last month
- Python bindings for UCX☆140Updated 2 weeks ago
- Home for OctoML PyTorch Profiler☆114Updated 2 years ago
- Unified storage framework for the entire machine learning lifecycle☆155Updated last year
- The Triton backend for the ONNX Runtime.☆162Updated last week
- torch::deploy (multipy for non-torch uses) is a system that lets you get around the GIL problem by running multiple Python interpreters i…☆181Updated last month
- A performant, memory-efficient checkpointing library for PyTorch applications, designed with large, complex distributed workloads in mind…☆161Updated 2 weeks ago
- ☆31Updated 5 months ago
- TorchX is a universal job launcher for PyTorch applications. TorchX is designed to have fast iteration time for training/research and sup…☆392Updated this week
- Nod.ai 🦈 version of 👻 . You probably want to start at https://github.com/nod-ai/shark for the product and the upstream IREE repository …☆106Updated 8 months ago
- Python bindings for ggml☆146Updated last year
- FIL backend for the Triton Inference Server☆83Updated 3 weeks ago
- Python 3 Bindings for the NVIDIA Management Library☆140Updated last year
- ML/DL Math and Method notes☆63Updated last year
- 🐍 Python bidding for the Hora Approximate Nearest Neighbor Search Algorithm library☆72Updated 3 years ago
- A user-friendly tool chain that enables the seamless execution of ONNX models using JAX as the backend.☆123Updated 2 weeks ago
- Demo of the unit_scaling library, showing how a model can be easily adapted to train in FP8.☆46Updated last year
- 🏙 Interactive performance profiling and debugging tool for PyTorch neural networks.☆64Updated 8 months ago
- ☆21Updated 7 months ago
- Productionize machine learning predictions, with ONNX or without☆66Updated last year
- Triton Model Navigator is an inference toolkit designed for optimizing and deploying Deep Learning models with a focus on NVIDIA GPUs.☆211Updated 5 months ago
- experiments with inference on llama☆104Updated last year
- Distributed XGBoost on Ray☆149Updated last year