InftyAI / Awesome-LLMOps
🎉 An awesome & curated list of best LLMOps tools.
☆42Updated this week
Alternatives and similar repositories for Awesome-LLMOps:
Users that are interested in Awesome-LLMOps are comparing it to the libraries listed below
- ☸️ Easy, advanced inference platform for large language models on Kubernetes. 🌟 Star to support our work!☆92Updated this week
- A diverse, simple, and secure all-in-one LLMOps platform☆101Updated 5 months ago
- 💫 A lightweight p2p-based cache system for model distributions on Kubernetes. Reframing now to make it an unified cache system with POSI…☆20Updated 3 months ago
- 🧯 Kubernetes coverage for fault awareness and recovery, works for any LLMOps, MLOps, AI workloads.☆26Updated 2 months ago
- The main purpose of runtime copilot is to assist with node runtime management tasks such as configuring registries, upgrading versions, i…☆12Updated last year
- A simple service that integrates vLLM with Ray Serve for fast and scalable LLM serving.☆63Updated 11 months ago
- ☆12Updated last year
- A QA system based on k8s-specific knowledge build on ChatGLM2-6B, serving by Ray.☆10Updated last year
- Knowledge for GPTScript☆29Updated 4 months ago
- knavigator is a development, testing, and optimization toolkit for AI/ML scheduling systems at scale on Kubernetes.☆62Updated last month
- GenAI inference performance benchmarking tool☆19Updated this week
- d.run website☆13Updated this week
- OpenAI compatible API for open source LLMs☆15Updated last year
- ☆15Updated last week
- This is a landscape of the infrastructure that powers the generative AI ecosystem☆138Updated 4 months ago
- Open-source observability for your LLM application.☆51Updated 2 months ago
- Automated pull requests reviewing and issues triaging with ChatGPT.☆70Updated last year
- Kubernetes APIServer 高性能代理组件,代理 APIServer 的 List 请求,其它类型的请求会直接反向代理到原生 APIServer。 CKube 还额外支持了分页、搜索和索引等功能。 并且,CKube 100% 兼容原生 kubectl 和 ku…☆19Updated 2 years ago
- ☆34Updated this week
- 📡 Deploy AI models and apps to Kubernetes without developing a hernia☆32Updated 9 months ago
- 🚢 Yet another operator for running large language models on Kubernetes with ease. Powered by Ollama! 🐫☆166Updated this week
- Extensible generative AI platform on Kubernetes with OpenAI-compatible APIs.☆63Updated this week
- MCP server connecting to Kubernetes☆77Updated 3 weeks ago
- API Extensions for core KubeVela.☆11Updated 3 weeks ago
- Manage kubernetes node-level kernel tuning ( using sysctl ).☆27Updated last week
- Self-host LLMs with vLLM and BentoML☆92Updated this week
- A federation scheduler for multi-cluster☆32Updated 2 weeks ago
- MCP Server for kubernetes management commands☆89Updated last week
- Resource Exporter for volcano scheduling, e.g. NUMA-Aware scheduling.☆15Updated 4 months ago