leimao / Simple-Inference-ServerLinks
Inference Server Implementation from Scratch for Machine Learning Models
☆24Updated 5 years ago
Alternatives and similar repositories for Simple-Inference-Server
Users that are interested in Simple-Inference-Server are comparing it to the libraries listed below
Sorting:
- An Open Convolutional Neural Network Framework in C++ From Scratch☆66Updated 4 years ago
- ResNet Implementation, Training, and Inference Using LibTorch C++ API☆42Updated last year
- torch::deploy (multipy for non-torch uses) is a system that lets you get around the GIL problem by running multiple Python interpreters i…☆182Updated 2 weeks ago
- Context Manager to profile the forward and backward times of PyTorch's nn.Module☆83Updated 2 years ago
- ONNX Command-Line Toolbox☆35Updated last year
- Training neural networks in TensorFlow 2.0 with 5x less memory☆137Updated 3 years ago
- Scailable ONNX python tools☆97Updated last year
- Customized matrix multiplication kernels☆57Updated 3 years ago
- A GPU performance profiling tool for PyTorch models☆22Updated 3 years ago
- Fairring (FAIR + Herring) is a plug-in for PyTorch that provides a process group for distributed training that outperforms NCCL at large …☆65Updated 3 years ago
- Highly optimized inference engine for Binarized Neural Networks☆251Updated last week
- implement AlexNet with C / convolutional nerual network / machine learning / computer vision☆196Updated 4 years ago
- Fast sparse deep learning on CPUs☆56Updated 3 years ago
- Inference framework for MoE layers based on TensorRT with Python binding☆41Updated 4 years ago
- The Triton backend for the PyTorch TorchScript models.☆168Updated last week
- int8_t and int16_t matrix multiply based on https://arxiv.org/abs/1705.01991☆74Updated 2 years ago
- PyTorch interface for the IPU☆181Updated 2 years ago
- ☆22Updated 6 years ago
- PyProf2: PyTorch Profiling tool☆82Updated 5 years ago
- Module 1 - Autodifferentiation☆22Updated last year
- DLPack for Tensorflow☆35Updated 5 years ago
- Distributed DataLoader For Pytorch Based On Ray☆24Updated 4 years ago
- PyTorch C++ API Documentation☆242Updated this week
- This is a Tensor Train based compression library to compress sparse embedding tables used in large-scale machine learning models such as …☆194Updated 3 years ago
- MONeT framework for reducing memory consumption of DNN training☆174Updated 4 years ago
- Manually implemented quantization-aware training☆23Updated 3 years ago
- DiffQ performs differentiable quantization using pseudo quantization noise. It can automatically tune the number of bits used per weight …☆237Updated 2 years ago
- This repository contains the results and code for the MLPerf™ Training v0.7 benchmark.☆57Updated 2 years ago
- Research and development for optimizing transformers☆131Updated 4 years ago
- A tensor-aware point-to-point communication primitive for machine learning☆280Updated 2 weeks ago