IBM / onnx-mlir-serving
ONNX Serving is a project written with C++ to serve onnx-mlir compiled models with GRPC and other protocols.Benefiting from C++ implementation, ONNX Serving has very low latency overhead and high throughput. ONNX Servring provides dynamic batch aggregation and workers pool to fully utilize AI accelerators on the machine.
☆23Updated last year
Related projects ⓘ
Alternatives and complementary repositories for onnx-mlir-serving
- Open deep learning compiler stack for cpu, gpu and specialized accelerators☆33Updated last year
- ☆67Updated last year
- how to design cpu gemm on x86 with avx256, that can beat openblas.☆66Updated 5 years ago
- TensorFlow and TVM integration☆38Updated 4 years ago
- Standalone Flash Attention v2 kernel without libtorch dependency☆98Updated 2 months ago
- Benchmark scripts for TVM☆73Updated 2 years ago
- ☆23Updated 9 months ago
- This is a demo how to write a high performance convolution run on apple silicon☆52Updated 2 years ago
- ☆48Updated 8 months ago
- A standalone GEMM kernel for fp16 activation and quantized weight, extracted from FasterTransformer☆85Updated 8 months ago
- symmetric int8 gemm☆66Updated 4 years ago
- Conversions to MLIR EmitC☆124Updated 3 months ago
- Common libraries for PPL projects☆29Updated last month
- flexible-gemm conv of deepcore☆17Updated 4 years ago
- System for automated integration of deep learning backends.☆48Updated 2 years ago
- A sandbox for quick iteration and experimentation on projects related to IREE, MLIR, and LLVM☆55Updated 2 months ago
- Tencent NCNN with added CUDA support☆67Updated 3 years ago
- ☆22Updated last year
- Play with MLIR right in your browser☆124Updated last year
- An easy way to run, test, benchmark and tune OpenCL kernel files☆23Updated last year
- ☆12Updated 4 years ago
- MatMul Performance Benchmarks for a Single CPU Core comparing both hand engineered and codegen kernels.☆124Updated last year
- OneFlow Serving☆20Updated 9 months ago
- OneFlow->ONNX☆42Updated last year
- ☆55Updated 6 months ago
- IREE's PyTorch Frontend, based on Torch Dynamo.☆55Updated this week
- play gemm with tvm☆84Updated last year
- A language and compiler for irregular tensor programs.☆133Updated 6 months ago