llm-inference is a platform for publishing and managing llm inference, providing a wide range of out-of-the-box features for model deployment, such as UI, RESTful API, auto-scaling, computing resource management, monitoring, and more.
☆97May 17, 2024Updated 2 years ago
Alternatives and similar repositories for llm-inference
Users that are interested in llm-inference are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- The framework of training large language models，support lora, full parameters fine tune etc, define yaml to start training/fine tune of y…☆32Sep 19, 2024Updated 2 years ago
- This repository provides installation scripts and configuration files for deploying the CSGHub instance, includes Helm charts and Docker…☆22Updated this week
- The CSGHub SDK is a powerful Python client specifically designed to interact seamlessly with the CSGHub server. This toolkit is engineere…☆24Aug 29, 2026Updated last month
- LLM scheduler user interface☆21May 17, 2024Updated 2 years ago
- Your own personal AI team.☆23Updated this week
- 1-Click AI Models by DigitalOcean Gradient • AdDeploy popular AI models on DigitalOcean Gradient GPU virtual machines with just a single click. Zero configuration with optimized deployments.
- ☆17Mar 24, 2023Updated 3 years ago
- ☆20Sep 28, 2024Updated 2 years ago
- A side project that follows all the acceleration tricks in tinyllama, with the minimal modification to the huggingface transformers code.☆13Sep 2, 2024Updated 2 years ago
- RayLLM - LLMs on Ray (Archived). Read README for more info.☆1,259Mar 13, 2025Updated last year
- Inferflow is an efficient and highly configurable inference engine for large language models (LLMs).☆253Mar 15, 2024Updated 2 years ago
- a tool for gerenate dataset from doc☆13Jul 31, 2025Updated last year
- ScienceDiscovery is an all‑in‑one agentic workbench built specifically for scientific research.☆165Updated this week
- Elastic-Grok-Script-Plugin is a provider of Grok ElasticSearch plug-in☆12Dec 6, 2016Updated 9 years ago
- Large Language Model Onnx Inference Framework☆35Nov 25, 2025Updated 10 months ago
- Managed Database hosting by DigitalOcean • AdPostgreSQL, MySQL, MongoDB, Kafka, Valkey, and OpenSearch available. Automatically scale up storage and focus on building your apps.
- Codes for our paper "AgentMonitor: A Plug-and-Play Framework for Predictive and Secure Multi-Agent Systems"☆14Dec 13, 2024Updated last year
- Track, search and filter issue status history in Redmine 5.x and 6.x☆11Aug 6, 2026Updated 2 months ago
- janusgraph data import☆10Jun 17, 2022Updated 4 years ago
- Compare different hardware platforms via the Roofline Model for LLM inference tasks.☆126Mar 13, 2024Updated 2 years ago
- Scapytain is a web application that enables you to store, organise and run test campaigns on top of Scapy.☆20Jun 19, 2018Updated 8 years ago
- The UnisonAI Multi-Agent Framework built on custom workflow which allows ai agents to talk together and provides a flexible and extensibl…☆23Feb 24, 2026Updated 7 months ago
- An open-source project to enable people to create and maintain suites of API's.☆17Feb 4, 2025Updated last year
- ☆75Mar 26, 2025Updated last year
- 官方transformers源码解析。AI大模型时代，pytorch、transformer是新操作系统，其他都是运行在其上面的软件。☆16Sep 25, 2023Updated 3 years ago
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- AxisRL is an agentic RL post-training framework built on SGLang rollout, Megatron training, and real-world agent workflows.☆612Aug 3, 2026Updated 2 months ago
- Vector Bazel Rules and Toolchains☆16Mar 2, 2026Updated 7 months ago
- Azure Machine Learning - MLOps Python SDKv2☆10Jul 24, 2023Updated 3 years ago
- Dynamic Memory Management for Serving LLMs without PagedAttention☆525Aug 24, 2026Updated last month
- PWC Artifacts☆11Mar 24, 2015Updated 11 years ago
- ☆172Oct 9, 2024Updated 2 years ago
- Mathematical expression evaluator with just in time code generation.☆12Apr 7, 2013Updated 13 years ago
- Kubernetes device plugin for Biren GPU☆11Oct 17, 2024Updated 2 years ago
- Redmine plugin to customize shown columns in subtasks and related issues on issue page☆13May 9, 2020Updated 6 years ago
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- 从jieba分词到BERT-wwm，一步步带你进入中文NLP的世界☆15Sep 1, 2022Updated 4 years ago
- ☆74Sep 22, 2026Updated 3 weeks ago
- Distributed SDDMM Kernel☆13Jul 8, 2022Updated 4 years ago
- Selection-based Question Answering☆14Feb 7, 2018Updated 8 years ago
- Pretrain, finetune and serve LLMs on Intel platforms with Ray☆129Sep 23, 2025Updated last year
- Parallel processing with sequential output, respecting order of input☆10Feb 20, 2023Updated 3 years ago
- A proxy behind nginx while before kibana (4.x, 5.x) to provide data isolation for different users☆23May 6, 2019Updated 7 years ago