liu-mengyang / rust-mmdeploy

Related projects ⓘ

Alternatives and complementary repositories for rust-mmdeploy

• triton-inference-server / paddlepaddle_backend
  ☆32Updated 9 months ago
• triple-Mu / TensorRT2ONNX
  A tool convert TensorRT engine/plan to a fake onnx
  ☆37Updated 2 years ago
• torchpipe / torchpipe
  Serving Inside Pytorch
  ☆145Updated this week
• FeiGeChuanShu / trt2023
  NVIDIA TensorRT Hackathon 2023复赛选题：通义千问Qwen-7B用TensorRT-LLM模型搭建及优化
  ☆40Updated last year
• inisis / OnnxSlim
  A Toolkit to Help Optimize Onnx Model
  ☆81Updated this week
• InternLM / turbomind
  ☆35Updated 2 weeks ago
• triton-inference-server / dali_backend
  The Triton backend that allows running GPU-accelerated data pre-processing pipelines implemented in DALI's python API.
  ☆125Updated 2 weeks ago
• Oneflow-Inc / oneflow_convert
  OneFlow->ONNX
  ☆42Updated last year
• BBuf / megatron-lm-parallel-group-playground
  ☆13Updated 7 months ago
• Oldpan / DeployIsAllYouNeed
  ☆118Updated last year
• SkyworkAI / vllm
  A high-throughput and memory-efficient inference and serving engine for LLMs
  ☆15Updated 5 months ago
• nanmi / RT-DETR-Deploy
  Deploy RT-EDTR with onnx from paddlepaddle framwork and graph cut
  ☆28Updated last year
• MegEngine / mgeconvert
  MegEngine到其他框架的转换器
  ☆67Updated last year
• RangiLyu / llama.mmengine
  Training LLaMA language model with MMEngine! It supports LoRA fine-tuning!
  ☆40Updated last year
• tlc-pack / libflash_attn
  Standalone Flash Attention v2 kernel without libtorch dependency
  ☆98Updated 2 months ago
• Oneflow-Inc / vision
  Datasets, Transforms and Models specific to Computer Vision
  ☆83Updated last year
• tsingmicro-toolchain / OnnxSlim
  A Toolkit to Help Optimize Large Onnx Model
  ☆149Updated 6 months ago
• YH-Wu / Triton-Inference-Server-on-Kubernetes
  ☆30Updated 2 years ago
• TRT2022 / trtllm-llama
  ☢️ TensorRT 2023复赛——基于TensorRT-LLM的Llama模型推断加速优化
  ☆44Updated last year
• Oneflow-Inc / models
  Models and examples built with OneFlow
  ☆96Updated last month
• Oneflow-Inc / serving
  OneFlow Serving
  ☆20Updated 9 months ago
• ICT-ANS / StarLight
  ☆59Updated 4 months ago
• DefTruth / Awesome-Diffusion-Inference
  📒A small curated list of Awesome Diffusion Inference Papers with codes.
  ☆96Updated this week
• OpenBMB / cpm_kernels
  ☆23Updated last year
• luchangli03 / onnxsim_large_model
  simplify >2GB large onnx model
  ☆44Updated 8 months ago
• PaddlePaddle / PaDiff
  Paddle Automatically Diff Precision Toolkits.
  ☆47Updated 7 months ago
• TRT2022 / ControlNet_TensorRT
  天池 NVIDIA TensorRT Hackathon 2023 —— 生成式AI模型优化赛 初赛第三名方案
  ☆47Updated last year
• AXERA-TECH / ax-llm
  Explore LLM model deployment based on AXera's AI chips
  ☆54Updated last week
• Bruce-Lee-LY / flash_attention_inference
  Performance of the C++ interface of flash attention and flash attention v2 in large language model (LLM) inference scenarios.
  ☆29Updated 2 months ago
• openmlsys / openmlsys-cuda
  Tutorials for writing high-performance GPU operators in AI frameworks.
  ☆123Updated last year