OpenCSGs / llm-inference
llm-inference is a platform for publishing and managing llm inference, providing a wide range of out-of-the-box features for model deployment, such as UI, RESTful API, auto-scaling, computing resource management, monitoring, and more.
☆80Updated 11 months ago
Alternatives and similar repositories for llm-inference:
Users that are interested in llm-inference are comparing it to the libraries listed below
- The framework of training large language models，support lora, full parameters fine tune etc, define yaml to start training/fine tune of y…☆27Updated 7 months ago
- This repository provides installation scripts and configuration files for deploying the CSGHub instance, includes Helm charts and Docker…☆15Updated 2 weeks ago
- 配合 HAI Platform 使用的集成化用户界面☆49Updated last year
- bisheng-unstructured library☆46Updated last week
- The CSGHub SDK is a powerful Python client specifically designed to interact seamlessly with the CSGHub server. This toolkit is engineere…☆14Updated 2 weeks ago
- A Toolkit for Running On-device Large Language Models (LLMs) in APP☆72Updated 10 months ago
- ☆108Updated last year
- DashInfer is a native LLM inference engine aiming to deliver industry-leading performance atop various hardware architectures, including …☆245Updated this week
- AGI模块库架构图☆75Updated last year
- A high-throughput and memory-efficient inference and serving engine for LLMs☆135Updated 5 months ago
- Easy, fast, and cheap pretrain,finetune, serving for everyone☆295Updated last month
- ☆161Updated last month
- Imitate OpenAI with Local Models☆88Updated 8 months ago
- ☆32Updated last year
- Byzer-retrieval is a distributed retrieval system which designed as a backend for LLM RAG (Retrieval Augmented Generation). The system su…☆48Updated 2 months ago
- Implement OpenAI APIs and plugin-enabled ChatGPT with open source LLM and other models.☆120Updated 10 months ago
- GLM Series Edge Models☆137Updated 2 months ago
- A demo built on Megrez-3B-Instruct, integrating a web search tool to enhance the model's question-and-answer capabilities.☆37Updated 4 months ago
- XVERSE-MoE-A4.2B: A multilingual large language model developed by XVERSE Technology Inc.☆38Updated last year
- ☆29Updated 8 months ago
- Mixture-of-Experts (MoE) Language Model☆186Updated 8 months ago
- ☆323Updated 10 months ago
- Delta-CoMe can achieve near loss-less 1-bit compressin which has been accepted by NeurIPS 2024☆57Updated 5 months ago
- LLM 推理服务性能测试☆39Updated last year
- vLLM Documentation in Chinese Simplified / vLLM 中文文档☆63Updated 2 weeks ago
- A unified tool to generate fine-tuning datasets for LLMs, including questions, answers, and dialogues. ✨🤖📚💬☆58Updated last month
- 使用langchain进行任务规划，构建子任务的会话场景资源，通过MCTS任务执行器，来让每个子任务通过在上下文中资源，通过自身反思探索来获取自身对问题的最优答案；这种方式依赖模型的对齐偏好，我们在每种偏好上设计了一个工程框架，来完成自我对不同答案的奖励进行采样策略☆29Updated this week
- LLM scheduler user interface☆16Updated 11 months ago
- A open version Manus.☆58Updated last month
- GPT+神器，简单实用的一站式AGI架构，内置本地化，LLM模型，agent，矢量数据库，智能链chain☆48Updated last year