OpenCSGs / llm-inference
llm-inference is a platform for publishing and managing llm inference, providing a wide range of out-of-the-box features for model deployment, such as UI, RESTful API, auto-scaling, computing resource management, monitoring, and more.
☆80Updated 10 months ago
Alternatives and similar repositories for llm-inference:
Users that are interested in llm-inference are comparing it to the libraries listed below
- The framework of training large language models，support lora, full parameters fine tune etc, define yaml to start training/fine tune of y…☆25Updated 6 months ago
- ☆107Updated 11 months ago
- This repository provides installation scripts and configuration files for deploying the CSGHub instance, includes Helm charts and Docker…☆13Updated this week
- bisheng-unstructured library☆42Updated this week
- 配合 HAI Platform 使用的集成化用户界面☆47Updated last year
- Easy, fast, and cheap pretrain,finetune, serving for everyone☆291Updated last week
- The CSGHub SDK is a powerful Python client specifically designed to interact seamlessly with the CSGHub server. This toolkit is engineere…☆14Updated last week
- DashInfer is a native LLM inference engine aiming to deliver industry-leading performance atop various hardware architectures, including …☆238Updated 2 weeks ago
- AGI模块库架构图☆75Updated last year
- vLLM Documentation in Chinese Simplified / vLLM 中文文档☆55Updated 2 months ago
- A Toolkit for Running On-device Large Language Models (LLMs) in APP☆67Updated 8 months ago
- Convert files into markdown to help RAG or LLM understand, based on markitdown and MinerU, which could provide high quality pdf parser.☆64Updated 2 weeks ago
- Imitate OpenAI with Local Models☆88Updated 7 months ago
- run ChatGLM2-6B in BM1684X☆49Updated last year
- GLM Series Edge Models☆132Updated last month
- ☆256Updated last week
- 基于 Dify + Langfuse 的自动化评估服务☆45Updated 3 months ago
- Dingo: A Comprehensive Data Quality Evaluation Tool☆97Updated this week
- ☆59Updated 5 months ago
- SUS-Chat: Instruction tuning done right☆48Updated last year
- Implement OpenAI APIs and plugin-enabled ChatGPT with open source LLM and other models.☆120Updated 9 months ago
- The tool is used for building and driving workflows specifically tailored for AI initiatives. It can be used to construct AI agents.☆139Updated 8 months ago
- A high-throughput and memory-efficient inference and serving engine for LLMs☆132Updated 3 months ago
- A unified tool to generate fine-tuning datasets for LLMs, including questions, answers, and dialogues. ✨🤖📚💬☆57Updated 2 weeks ago
- 基于 Langchain，快速集成GLM-4 AllTools 功能的插件☆45Updated 8 months ago
- An Innovative Agent Framework Driven by KG Engine☆753Updated 2 months ago
- An easy-to-use framework for modular RAG☆341Updated this week
- LLM scheduler user interface☆15Updated 10 months ago
- Agentica: Effortlessly Build Intelligent, Reflective, and Collaborative Multimodal AI Agents! 轻松构建智能、具备反思能力、可协作的多模态AI Agent。☆145Updated 2 weeks ago
- ☆105Updated last year