InftyAI / Awesome-LLMOps
🎉 An awesome & curated list of best LLMOps tools.
☆63Updated this week
Alternatives and similar repositories for Awesome-LLMOps:
Users that are interested in Awesome-LLMOps are comparing it to the libraries listed below
- ☸️ Easy, advanced inference platform for large language models on Kubernetes. 🌟 Star to support our work!☆105Updated this week
- 🧯 Kubernetes coverage for fault awareness and recovery, works for any LLMOps, MLOps, AI workloads.☆28Updated 3 months ago
- 💫 A lightweight p2p-based cache system for model distributions on Kubernetes. Reframing now to make it an unified cache system with POSI…☆20Updated 3 months ago
- A diverse, simple, and secure all-in-one LLMOps platform☆101Updated 6 months ago
- ☆12Updated last year
- The main purpose of runtime copilot is to assist with node runtime management tasks such as configuring registries, upgrading versions, i…☆12Updated last year
- Knowledge for GPTScript☆29Updated 4 months ago
- knavigator is a development, testing, and optimization toolkit for AI/ML scheduling systems at scale on Kubernetes.☆64Updated last week
- 📡 Deploy AI models and apps to Kubernetes without developing a hernia☆32Updated 10 months ago
- 🚢 Yet another operator for running large language models on Kubernetes with ease. Powered by Ollama! 🐫☆175Updated last week
- Extensible generative AI platform on Kubernetes with OpenAI-compatible APIs.☆63Updated this week
- A simple service that integrates vLLM with Ray Serve for fast and scalable LLM serving.☆65Updated 11 months ago
- Kubernetes APIServer 高性能代理组件,代理 APIServer 的 List 请求,其它类型的请求会直接反向代理到原生 APIServer。 CKube 还额外支持了分页、搜索和索引等功能。 并且,CKube 100% 兼容原生 kubectl 和 ku…☆19Updated 2 years ago
- d.run website☆13Updated this week
- An Operator for deployment and maintenance of NVIDIA NIMs and NeMo microservices in a Kubernetes environment.☆88Updated this week
- A QA system based on k8s-specific knowledge build on ChatGLM2-6B, serving by Ray.☆10Updated last year
- This is a landscape of the infrastructure that powers the generative AI ecosystem☆139Updated 5 months ago
- Large language model fine-tuning capabilities based on cloud native and distributed computing.☆92Updated last year
- LeaderWorkerSet: An API for deploying a group of pods as a unit of replication☆339Updated this week
- Manage kubernetes node-level kernel tuning ( using sysctl ).☆27Updated 3 weeks ago
- API Extensions for core KubeVela.☆13Updated last month
- WG Serving☆20Updated last month
- GenAI inference performance benchmarking tool☆20Updated this week
- ☆15Updated 2 weeks ago
- Kuberentes LangChain Agent - Interact with Kubernetes Clusters using LLMs☆26Updated last year
- MCP server connecting to Kubernetes☆122Updated this week
- ☆94Updated 2 months ago