Tlntin / ChatGLM2-6B-TensorRTLinks
☆90Updated 2 years ago
Alternatives and similar repositories for ChatGLM2-6B-TensorRT
Users that are interested in ChatGLM2-6B-TensorRT are comparing it to the libraries listed below
Sorting:
- run ChatGLM2-6B in BM1684X☆49Updated last year
- ☢️ TensorRT 2023复赛——基于TensorRT-LLM的Llama模型推断加速优化☆50Updated last year
- 纯c++的全平台llm加速库,支持python调用,支持baichuan, glm, llama, moss基座,手机端流畅运行chatglm-6B级模型单卡可达10000+token / s,☆45Updated last year
- ☆27Updated 8 months ago
- Compare multiple optimization methods on triton to imporve model service performance☆52Updated last year
- Another ChatGLM2 implementation for GPTQ quantization☆54Updated last year
- export llama to onnx☆130Updated 7 months ago
- llm-export can export llm model to onnx.☆301Updated 6 months ago
- llama inference for tencentpretrain☆98Updated 2 years ago
- 使用sentencepiece中BPE训练中文词表,并在transformers中进行使用。☆118Updated 2 years ago
- A more efficient GLM implementation!☆55Updated 2 years ago
- ☆49Updated 9 months ago
- 高性能文本 Tokenizer 库☆29Updated last year
- simplify >2GB large onnx model☆61Updated 8 months ago
- ☆52Updated 2 years ago
- ChatGLM2-6B微调, SFT/LoRA, instruction finetune☆108Updated 2 years ago
- ☆83Updated last year
- ☆26Updated last year
- baichuan and baichuan2 finetuning and alpaca finetuning☆32Updated 4 months ago
- A high-throughput and memory-efficient inference and serving engine for LLMs☆136Updated 7 months ago
- Transformer related optimization, including BERT, GPT☆17Updated 2 years ago
- 天池 NVIDIA TensorRT Hackathon 2023 —— 生成式AI模型优化赛 初赛第三名方案☆49Updated last year
- qwen models finetuning☆101Updated 4 months ago
- Kanchil(鼷鹿)是世界上最小的偶蹄目动物,这个开源项目意在探索小模型(6B以下)是否也能具备和人类偏好对齐的能力。☆112Updated 2 years ago
- chatglm多gpu用deepspeed和☆407Updated last year
- 大模型部署实战:TensorRT-LLM, Triton Inference Server, vLLM☆26Updated last year
- Transformer related optimization, including BERT, GPT☆39Updated 2 years ago
- NVIDIA TensorRT Hackathon 2023复赛选题:通义千问Qwen-7B用TensorRT-LLM模型搭建及优化☆42Updated last year
- ☆307Updated 2 years ago
- 一个基于HuggingFace开发的大语言模型训练、测试工具。支持各模型的webui、终端预测,低参数量及全参数模型训练(预训练、SFT、RM、PPO、DPO)和融合、量化。☆218Updated last year