sophgo / ChatGLM2-TPULinks
run ChatGLM2-6B in BM1684X
☆50Updated last year
Alternatives and similar repositories for ChatGLM2-TPU
Users that are interested in ChatGLM2-TPU are comparing it to the libraries listed below
Sorting:
- ☢️ TensorRT 2023复赛——基于TensorRT-LLM的Llama模型推断加速优化☆50Updated 2 years ago
- NVIDIA TensorRT Hackathon 2023复赛选题：通义千问Qwen-7B用TensorRT-LLM模型搭建及优化☆43Updated 2 years ago
- ☆90Updated 2 years ago
- 天池 NVIDIA TensorRT Hackathon 2023 —— 生成式AI模型优化赛 初赛第三名方案☆50Updated 2 years ago
- ☆52Updated last year
- simplify >2GB large onnx model☆69Updated last year
- 大模型部署实战：TensorRT-LLM, Triton Inference Server, vLLM☆26Updated last year
- 基于MNN-llm的安卓手机部署大语言模型：Qwen1.5-0.5B-Chat☆86Updated last year
- qwen2 and llama3 cpp implementation☆48Updated last year
- Compare multiple optimization methods on triton to imporve model service performance☆52Updated last year
- Run generative AI models in sophgo BM1684X/BM1688☆254Updated this week
- Large Language Model Onnx Inference Framework☆36Updated last week
- export llama to onnx☆137Updated 11 months ago
- stable diffusion using mnn☆67Updated 2 years ago
- 大模型API性能指标比较 - 深入分析TTFT、TPS等关键指标☆19Updated last year
- 纯c++的全平台llm加速库，支持python调用，支持baichuan, glm, llama, moss基座，手机端流畅运行chatglm-6B级模型单卡可达10000+token / s，☆45Updated 2 years ago
- ☆125Updated last year
- ☆26Updated 2 years ago
- Transformer related optimization, including BERT, GPT☆17Updated 2 years ago
- Serving Inside Pytorch☆165Updated 2 weeks ago
- llm-export can export llm model to onnx.☆333Updated last month
- ☆120Updated 2 years ago
- ggml学习笔记，ggml是一个机器学习的推理框架☆18Updated last year
- Explore LLM model deployment based on AXera's AI chips☆130Updated last week
- ☆65Updated last week
- A high performance, high expansion, easy to use framework for AI application. 为AI应用的开发者提供一套统一的高性能、易用的编程框架，快速基于AI全栈服务、开发跨端边云的AI行业应用，支持GPU，…☆160Updated last year
- ☆42Updated last year
- Another ChatGLM2 implementation for GPTQ quantization☆54Updated 2 years ago
- ☆60Updated last year
- GLM Series Edge Models☆153Updated 5 months ago