Alternatives and similar repositories for benchmark-pytorch2.0-with-nebullvm:

Users that are interested in benchmark-pytorch2.0-with-nebullvm are comparing it to the libraries listed below

• onnx / neural-compressor
  Model compression for ONNX
  ☆81Updated 2 months ago
• PINTO0309 / sne4onnx
  A very simple tool for situations where optimization with onnx-simplifier would exceed the Protocol Buffers upper file size limit of 2GB,…
  ☆15Updated 8 months ago
• philschmid / optimum-static-quantization
  ☆28Updated last year
• FrancescoSaverioZuppichini / dynamic-batching-asyncio
  ☆31Updated last year
• NVIDIA-AI-IOT / whisper_trt
  A project that optimizes Whisper for low latency inference using NVIDIA TensorRT
  ☆69Updated 3 months ago
• fpaupier / gRPC-multiprocessing
  A boilerplate to use multiprocessing for your gRPC server in your Python project
  ☆25Updated 3 years ago
• triton-inference-server / tensorrt_backend
  The Triton backend for TensorRT.
  ☆68Updated this week
• PINTO0309 / whisper-onnx-tensorrt
  ONNX and TensorRT implementation of Whisper
  ☆61Updated last year
• triton-inference-server / onnxruntime_backend
  The Triton backend for the ONNX Runtime.
  ☆136Updated this week
• jundaf2 / INT8-Flash-Attention-FMHA-Quantization
  ☆157Updated last year
• mtszkw / fast-torch
  Comparing PyTorch, JIT and ONNX for inference with Transformers
  ☆17Updated 3 years ago
• dusty-nv / NanoDB
  Zero-copy multimodal vector DB with CUDA and CLIP/SigLIP
  ☆41Updated 7 months ago
• triple-Mu / TensorRT2ONNX
  A tool convert TensorRT engine/plan to a fake onnx
  ☆37Updated 2 years ago
• triton-inference-server / triton_cli
  Triton CLI is an open source command line interface that enables users to create, deploy, and profile models served by the Triton Inferen…
  ☆52Updated this week
• leimao / PyTorch-Eager-Mode-Quantization-TensorRT-Acceleration
  TensorRT Acceleration for PyTorch Native Eager Mode Quantization Models
  ☆14Updated 5 months ago
• roboflow / deploy-models-with-grpc-pytorch-asyncio
  Article about deploying machine learning models using grpc, pytorch and asyncio
  ☆27Updated 2 years ago
• lessw2020 / transformer_central
  Various transformers for FSDP research
  ☆34Updated 2 years ago
• quic / efficient-transformers
  This library empowers users to seamlessly port pretrained models and checkpoints on the HuggingFace (HF) hub (developed using HF transfor…
  ☆57Updated this week
• neuralmagic / nm-vllm
  A high-throughput and memory-efficient inference and serving engine for LLMs
  ☆257Updated 3 months ago
• scailable / sclblonnx
  Scailable ONNX python tools
  ☆96Updated 2 months ago
• wejoncy / QLLM
  A general 2-8 bits quantization toolbox with GPTQ/AWQ/HQQ, and export to onnx/onnx-runtime easily.
  ☆153Updated 3 months ago
• k9ele7en / Triton-TensorRT-Inference-CRAFT-pytorch
  Advanced inference pipeline using NVIDIA Triton Inference Server for CRAFT Text detection (Pytorch), included converter from Pytorch -> O…
  ☆32Updated 3 years ago
• aniketmaurya / stable_diffusion_inference
  Simple and easy stable diffusion inference with LightningModule on GPU, CPU and MPS (Possibly all devices supported by Lightning).
  ☆17Updated last year
• Lightning-Universe / lightning-ColossalAI
  Large Scale Distributed Model Training strategy with Colossal AI and Lightning AI
  ☆58Updated last year
• facebookresearch / NasRec
  NASRec Weight Sharing Neural Architecture Search for Recommender Systems
  ☆29Updated last year
• ENOT-AutoDL / ONNX-Runtime-with-TensorRT-and-OpenVINO
  Docker scripts for building ONNX Runtime with TensorRT and OpenVINO in manylinux environment
  ☆23Updated last year
• NetEase-FuXi / EETQ
  Easy and Efficient Quantization for Transformers
  ☆191Updated last month
• dusty-nv / clip_trt
  CLIP and SigLIP models optimized with TensorRT with a Transformers-like API
  ☆19Updated 3 months ago
• fidelity / stoke
  A lightweight wrapper for PyTorch that provides a simple declarative API for context switching between devices, distributed modes, mixed-…
  ☆66Updated last year