pandada8 / llm-inference-benchmarkLinks
LLM 推理服务性能测试
☆43Updated last year
Alternatives and similar repositories for llm-inference-benchmark
Users that are interested in llm-inference-benchmark are comparing it to the libraries listed below
Sorting:
- LLM101n: Let's build a Storyteller 中文版☆132Updated last year
- DashInfer is a native LLM inference engine aiming to deliver industry-leading performance atop various hardware architectures, including …☆265Updated 2 months ago
- vLLM Documentation in Chinese Simplified / vLLM 中文文档☆110Updated last month
- ☆174Updated last week
- ☆63Updated 3 weeks ago
- LLM Inference benchmark☆426Updated last year
- 通义千问VLLM推理部署DEMO☆608Updated last year
- Accelerate inference without tears☆327Updated last week
- ☆50Updated 11 months ago
- 一些大语言模型和多模态模型的生态,主要包括跨模态搜索、投机解码、QAT量化、多模态量化、ChatBot、OCR☆191Updated last month
- ☆115Updated 11 months ago
- 训练一个对中文支持更好的LLaVA模型,并开源训练代码和数据。☆73Updated last year
- FlagScale is a large model toolkit based on open-sourced projects.☆358Updated last week
- ☆354Updated last year
- 欢迎来到 "LLM-travel" 仓库!探索大语言模型(LLM)的奥秘 🚀。致力于深入理解、探讨以及实现与大模型相关的各种技术、原理和应用。☆343Updated last year
- unify-easy-llm(ULM)旨在打造一个简易的一键式大模型训练工具,支持Nvidia GPU、Ascend NPU等不同硬件以及常用的大模型。☆57Updated last year
- A flexible and efficient training framework for large-scale alignment tasks☆428Updated last week
- ☆232Updated last year
- ☆52Updated 2 years ago
- ☆503Updated 3 weeks ago
- InternEvo is an open-sourced lightweight training framework aims to support model pre-training without the need for extensive dependencie…☆407Updated last month
- Alpaca Chinese Dataset -- 中文指令微调数据集☆214Updated last year
- ☆355Updated this week
- This is a user guide for the MiniCPM and MiniCPM-V series of small language models (SLMs) developed by ModelBest. “面壁小钢炮” focuses on achi…☆293Updated 3 months ago
- 中文大模型微调(LLM-SFT), 数学指令数据集MWP-Instruct, 支持模型(ChatGLM-6B, LLaMA, Bloom-7B, baichuan-7B), 支持(LoRA, QLoRA, DeepSpeed, UI, TensorboardX), 支持(微…☆211Updated last year
- Inference code for LLaMA models☆123Updated 2 years ago
- NLP 项目记录档案☆59Updated 5 months ago
- A high-throughput and memory-efficient inference and serving engine for LLMs☆137Updated 10 months ago
- LLaMA Factory Document☆151Updated 3 weeks ago
- Community maintained hardware plugin for vLLM on Ascend☆1,179Updated last week