DataXujing / TensorRT-LLM-ChatGLM3
大模型部署实战：TensorRT-LLM, Triton Inference Server, vLLM
☆26Updated 11 months ago
Alternatives and similar repositories for TensorRT-LLM-ChatGLM3:
Users that are interested in TensorRT-LLM-ChatGLM3 are comparing it to the libraries listed below
- NVIDIA TensorRT Hackathon 2023复赛选题：通义千问Qwen-7B用TensorRT-LLM模型搭建及优化☆41Updated last year
- 天池 NVIDIA TensorRT Hackathon 2023 —— 生成式AI模型优化赛 初赛第三名方案☆48Updated last year
- ☢️ TensorRT 2023复赛——基于TensorRT-LLM的Llama模型推断加速优化☆44Updated last year
- ffmpeg+cuvid+tensorrt+multicamera☆12Updated last month
- ☆120Updated last year
- HunyuanDiT with TensorRT and libtorch☆17Updated 8 months ago
- run ChatGLM2-6B in BM1684X☆49Updated 11 months ago
- Large Language Model Onnx Inference Framework☆30Updated last month
- Transformer related optimization, including BERT, GPT☆17Updated last year
- A high-throughput and memory-efficient inference and serving engine for LLMs☆16Updated 8 months ago
- llm deploy project based onnx.☆32Updated 4 months ago
- ☆10Updated 7 months ago
- ☆23Updated last year
- TensorRT简明教程☆26Updated 3 years ago
- 🐱 ncnn int8 模型量化评估☆12Updated 2 years ago
- qwen2 and llama3 cpp implementation☆40Updated 8 months ago
- An onnx-based quantitation tool.☆71Updated last year
- A tool convert TensorRT engine/plan to a fake onnx☆37Updated 2 years ago
- 使用 CUDA C++ 实现的 llama 模型推理框架☆45Updated 3 months ago
- SAM and lama inpaint，包含QT的GUI交互界面，实现了交互式可实时显示结果的画点、画框进行SAM，然后通过进行Inpaint，具体操作看readme里的视频。☆45Updated last year
- ☆90Updated last year
- ☆39Updated 3 months ago
- ☆14Updated 10 months ago
- Collection of blogs on AI development☆18Updated 3 months ago
- ☆20Updated last year
- Compare multiple optimization methods on triton to imporve model service performance☆49Updated last year
- ☆19Updated last year
- ☆27Updated 3 months ago