gameofdimension / vllm
A high-throughput and memory-efficient inference and serving engine for LLMs
☆9Updated last year
Alternatives and similar repositories for vllm:
Users that are interested in vllm are comparing it to the libraries listed below
- Large-scale exact string matching tool☆15Updated 2 months ago
- zero零训练llm调参☆31Updated last year
- A more efficient GLM implementation!☆55Updated last year
- 在kaggle部署ChatGLM API，和ChatGPT api使用相同的调用方式☆14Updated last year
- A prompt set of ChatGLM-6B☆14Updated last year
- 纯c++的全平台llm加速库，支持python调用，支持baichuan, glm, llama, moss基座，手机端流畅运行chatglm-6B级模型单卡可达10000+token / s，☆45Updated last year
- Finetune Llama 3, Mistral & Gemma LLMs 2-5x faster with 80% less memory☆25Updated 8 months ago
- qwen2 and llama3 cpp implementation☆38Updated 7 months ago
- XVERSE-MoE-A4.2B: A multilingual large language model developed by XVERSE Technology Inc.☆36Updated 8 months ago
- GRAIN: Gradient-based Intra-attention Pruning on Pre-trained Language Models☆17Updated last year
- Service for Bert model to Vector. 高效的文本转向量(Text-To-Vector)服务，支持GPU多卡、多worker、多客户端调用，开箱即用 。☆10Updated 2 years ago
- GPT+神器，简单实用的一站式AGI架构，内置本地化，LLM模型，agent，矢量数据库，智能链chain☆48Updated last year
- A light proxy solution for HuggingFace hub.☆46Updated last year
- the newest version of llama3，source code explained line by line using Chinese☆22Updated 9 months ago
- AGM阿格姆：AI基因图谱模型，从token-weight权重微粒角度，探索AI模型，GPT\LLM大模型的内在运作机制。☆28Updated last year
- Its an open source LLM based on MOE Structure.☆57Updated 6 months ago
- 演示 vllm 对中文大语言模型的神奇效果☆31Updated last year
- share data， prompt data , pretraining data☆35Updated last year
- aigc evals☆10Updated last year
- 千问14B和7B的逐行解释☆53Updated last year
- 基于Paddle进行语义检索并部署 上线，支持多语言 This code is based on Paddle to do a semantic search, and deploy it. Multilingual support☆11Updated 2 years ago
- Whisper in TensorRT-LLM☆15Updated last year
- qwen create prompt for sdxl☆31Updated last year
- ☆24Updated 3 months ago
- 百度QA100万数据集☆47Updated last year
- XVERSE-MoE-A36B: A multilingual large language model developed by XVERSE Technology Inc.☆37Updated 4 months ago
- TianMu: A modern AI tool with multi-platform support, markdown support, multimodal, continuous conversation, and customizable commands. 一…☆84Updated last year
- Another ChatGLM2 implementation for GPTQ quantization☆54Updated last year