MegEngine / InferLLMLinks
a lightweight LLM model inference framework
☆748Updated last year
Alternatives and similar repositories for InferLLM
Users that are interested in InferLLM are comparing it to the libraries listed below
Sorting:
- C++ implementation of Qwen-LM☆616Updated last year
- llm deploy project based mnn. This project has merged into MNN.☆1,615Updated last year
- llm-export can export llm model to onnx.☆344Updated 3 months ago
- 支持中文场景的的小语言模型 llama2.c-zh☆150Updated last year
- ☆437Updated 4 months ago
- Efficient AI Inference & Serving☆480Updated 2 years ago
- DashInfer is a native LLM inference engine aiming to deliver industry-leading performance atop various hardware architectures, including …☆274Updated 5 months ago
- LLaMa/RWKV onnx models, quantization and testcase☆367Updated 2 years ago
- torch_musa is an open source repository based on PyTorch, which can make full use of the super computing power of MooreThreads graphics c…☆471Updated last week
- ☆523Updated last week
- RTP-LLM: Alibaba's high-performance LLM inference engine for diverse applications.☆1,037Updated this week
- fastllm是后端无依赖的高性能大模型推理库。同时支持张量并行推理稠密模型和混合模式推理MOE模型，任意10G以上显卡即可推理满血DeepSeek。双路9004/9005服务器+单显卡部署DeepSeek满血满精度原版模型，单并发20tps；INT4量化模型单并发30tp…☆4,132Updated last week
- export llama to onnx☆137Updated last year
- Yuan 2.0 Large Language Model☆690Updated last year
- ☆182Updated this week
- LLM Inference benchmark☆433Updated last year
- Accelerate inference without tears☆372Updated last week
- optimized BERT transformer inference on NVIDIA GPU. https://arxiv.org/abs/2210.03052☆476Updated last year
- Play LLaMA2 (official / 中文版 / INT4 / llama2.cpp) Together! ONLY 3 STEPS! ( non GPU / 5GB vRAM / 8~14GB vRAM)☆542Updated 2 years ago
- ☆130Updated last year
- C++ implementation of ChatGLM-6B & ChatGLM2-6B & ChatGLM3 & GLM4(V)☆2,967Updated last year
- 中文Mixtral混合专家大模型（Chinese Mixtral MoE LLMs）☆609Updated last year
- Low-bit LLM inference on CPU/NPU with lookup table☆915Updated 7 months ago
- The official repo of Aquila2 series proposed by BAAI, including pretrained & chat large language models.☆446Updated last year
- Ascend PyTorch adapter (torch_npu). Mirror of https://gitee.com/ascend/pytorch☆483Updated this week
- PaddlePaddle custom device implementaion. (『飞桨』自定义硬件接入实现)☆101Updated this week
- INT4/INT5/INT8 and FP16 inference on CPU for RWKV language model☆1,563Updated 10 months ago
- Transformer related optimization, including BERT, GPT☆39Updated 2 years ago
- XVERSE-13B: A multilingual large language model developed by XVERSE Technology Inc.☆645Updated last year
- ☆457Updated 2 years ago