yvonwin / qwen2.cpp
qwen2 and llama3 cpp implementation
☆34Updated 3 months ago
Related projects: ⓘ
- NVIDIA TensorRT Hackathon 2023复赛选题:通义千问Qwen-7B用TensorRT-LLM模型搭建及优化☆39Updated 10 months ago
- Whisper in TensorRT-LLM☆14Updated 11 months ago
- 基于MNN-llm的安卓手机部署大语言模型:Qwen1.5-0.5B-Chat☆40Updated 5 months ago
- run ChatGLM2-6B in BM1684X☆48Updated 6 months ago
- ☢️ TensorRT 2023复赛——基于TensorRT-LLM的Llama模型推断加速优化☆40Updated 11 months ago
- ☆123Updated 9 months ago
- 大模型部署实战:TensorRT-LLM, Triton Inference Server, vLLM☆25Updated 6 months ago
- ☆17Updated 3 months ago
- ☆90Updated last year
- simplify >2GB large onnx model☆41Updated 6 months ago
- Transformer related optimization, including BERT, GPT☆17Updated last year
- ggml学习笔记,ggml是一个机器学习的推理框架☆11Updated 5 months ago
- 使用onnxruntime部署实时视频帧插值,包含C++和Python两个版本的程序☆19Updated 7 months ago
- 深度学习模型加解密工具☆35Updated last year
- 天池 NVIDIA TensorRT Hackathon 2023 —— 生成式AI模型优化赛 初赛第三名方案☆47Updated last year
- A more efficient GLM implementation!☆54Updated last year
- Another ChatGLM2 implementation for GPTQ quantization☆54Updated 11 months ago
- 纯c++的全平台llm加速库,支持python调用,支持baichuan, glm, llama, moss基座,手机端流畅运行chatglm-6B级模型单卡可达10000+token / s,☆44Updated last year
- ☆27Updated last month
- Explore LLM model deployment based on AXera's AI chips☆48Updated 2 weeks ago
- deploy onnx models with TensorRT and LibTorch☆16Updated 2 years ago
- a naive example of LivePortrait infer by ncnn☆30Updated last month
- libvits-ncnn is an ncnn implementation of the VITS library that enables cross-platform GPU-accelerated speech synthesis.🎙️💻☆55Updated last year
- 使用ONNXRuntime部署百度PaddleSeg发布的实时人像抠图模型PP-MattingV2,一共包含18个onnx模型,依然是包含C++和Python两个版本的程序☆26Updated last year
- A high-throughput and memory-efficient inference and serving engine for LLMs☆15Updated 3 months ago
- Port of Facebook's LLaMA model in C/C++☆66Updated this week
- segment-anything based mnn☆33Updated 9 months ago
- stable diffusion using mnn☆58Updated 11 months ago
- Music large model based on InternLM2-chat.☆21Updated last month
- the C++ version of Transformer with ncnn☆19Updated 3 years ago