chu-tianxiang / vllm-gptq
A high-throughput and memory-efficient inference and serving engine for LLMs
☆130Updated 7 months ago
Alternatives and similar repositories for vllm-gptq:
Users that are interested in vllm-gptq are comparing it to the libraries listed below
- A high-throughput and memory-efficient inference and serving engine for LLMs☆129Updated 2 months ago
- Imitate OpenAI with Local Models☆86Updated 5 months ago
- Generate multi-round conversation roleplay data based on self-instruct and evol-instruct.☆120Updated last month
- This is a text generation method which returns a generator, streaming out each token in real-time during inference, based on Huggingface/…☆96Updated 11 months ago
- Mixture-of-Experts (MoE) Language Model☆184Updated 5 months ago
- The official codes for "Aurora: Activating chinese chat capability for Mixtral-8x7B sparse Mixture-of-Experts through Instruction-Tuning"☆260Updated 9 months ago
- 旨在对当前主流LLM进行一个直观、具体、标准的评测☆92Updated last year
- A self-ailgnment method for role-play. Benchmark for role-play. Resources for "Large Language Models are Superpositions of All Characters…☆183Updated 8 months ago
- ☆122Updated last year
- train llama on a single A100 80G node using 🤗 transformers and 🚀 Deepspeed Pipeline Parallelism☆215Updated last year
- fastertransformer for codegeex model☆64Updated last year
- LongQLoRA: Extent Context Length of LLMs Efficiently☆163Updated last year
- DashInfer is a native LLM inference engine aiming to deliver industry-leading performance atop various hardware architectures, including …☆231Updated last week
- SUS-Chat: Instruction tuning done right☆48Updated last year
- ☆81Updated 9 months ago
- Open Source Text Embedding Models with OpenAI Compatible API☆145Updated 7 months ago
- SuperCLUE-Agent: 基于中文原生任务的Agent智能体核心能力测评基准☆81Updated last year
- 使用qlora对中文大语言模型进行微调，包含ChatGLM、Chinese-LLaMA-Alpaca、BELLE☆85Updated last year
- Comparison of Language Model Inference Engines☆204Updated 2 months ago
- ☆307Updated 7 months ago
- 骆驼QA，中文大语言阅读理解模型。☆74Updated last year
- Implementation of the LongRoPE: Extending LLM Context Window Beyond 2 Million Tokens Paper☆125Updated 7 months ago
- XVERSE-65B: A multilingual large language model developed by XVERSE Technology Inc.☆134Updated 10 months ago
- Light local website for displaying performances from different chat models.☆85Updated last year
- zero零训练llm调参☆31Updated last year
- ChatGLM2-6B微调, SFT/LoRA, instruction finetune☆105Updated last year
- deep learning☆150Updated 7 months ago
- Implement OpenAI APIs and plugin-enabled ChatGPT with open source LLM and other models.☆120Updated 8 months ago
- Official repository for LongChat and LongEval☆519Updated 8 months ago
- [EMNLP 2024] LongAlign: A Recipe for Long Context Alignment of LLMs☆241Updated 2 months ago