triton-inference-server / perf_analyzer
☆12Updated this week
Related projects: ⓘ
- A tool convert TensorRT engine/plan to a fake onnx☆37Updated last year
- 使用 cutlass 仓库在 ada 架构上实现 fp8 的 flash attention☆46Updated last month
- Standalone Flash Attention v2 kernel without libtorch dependency☆93Updated last week
- ☆32Updated 3 months ago
- A CUDA kernel for NHWC GroupNorm for PyTorch☆14Updated 2 weeks ago
- study of cutlass☆18Updated last year
- CUDA 8-bit Tensor Core Matrix Multiplication based on m16n16k16 WMMA API☆22Updated last year
- Nsight Compute in Docker☆11Updated 8 months ago
- Several optimization methods of half-precision general matrix vector multiplication (HGEMV) using CUDA core.☆40Updated last week
- Flash Attention in raw Cuda C beating PyTorch☆11Updated 4 months ago
- OneFlow Serving☆20Updated 7 months ago
- A standalone GEMM kernel for fp16 activation and quantized weight, extracted from FasterTransformer☆82Updated 6 months ago
- The Triton backend for TensorRT.☆59Updated last week
- Common source, scripts and utilities shared across all Triton repositories.☆62Updated last week
- ☆10Updated 3 years ago
- ☆17Updated 3 months ago
- HunyuanDiT with TensorRT and libtorch☆15Updated 3 months ago
- ☆22Updated last year
- Performance of the C++ interface of flash attention and flash attention v2 in large language model (LLM) inference scenarios.☆20Updated last week
- ☆13Updated 5 months ago
- ☆52Updated this week
- ☆33Updated last year
- ☆14Updated 8 months ago
- OneFlow->ONNX☆41Updated last year
- ☢️ TensorRT 2023复赛——基于TensorRT-LLM的Llama模型推断加速优化☆40Updated 11 months ago
- ☆15Updated last week
- NVIDIA TensorRT Hackathon 2023复赛选题:通义千问Qwen-7B用TensorRT-LLM模型搭建及优化☆39Updated 11 months ago
- ☆18Updated 5 months ago
- ☆10Updated 2 months ago
- ☆56Updated this week