Alternatives and similar repositories for llmos

Users that are interested in llmos are comparing it to the libraries listed below

• InftyAI / Awesome-LLMOps
  🎉 An awesome & curated list of best LLMOps tools.
  ☆167Updated last month
• InftyAI / llmaz
  ☸️ Easy, advanced inference platform for large language models on Kubernetes. 🌟 Star to support our work!
  ☆267Updated this week
• kubeagi / arcadia
  A diverse, simple, and secure all-in-one LLMOps platform
  ☆109Updated last year
• NVIDIA-AI-Blueprints / llm-router
  Route LLM requests to the best model for the task at hand.
  ☆125Updated 2 weeks ago
• docker / model-runner
  Docker Model Runner
  ☆218Updated last week
• makllama / makllama
  MaK(Mac+Kubernetes)llama - Running and orchestrating large language models (LLMs) on Kubernetes with macOS nodes.
  ☆43Updated last year
• obot-platform / obot
  Open-source MCP Gateway and AI Platform
  ☆445Updated this week
• NVIDIA / topograph
  A toolkit for discovering cluster network topology.
  ☆81Updated this week
• nekomeowww / ollama-operator
  🚢 Yet another operator for running large language models on Kubernetes with ease. Powered by Ollama! 🐫
  ☆223Updated this week
• llmariner / llmariner
  Extensible generative AI platform on Kubernetes with OpenAI-compatible APIs.
  ☆90Updated last month
• gpustack / gguf-parser-go
  Review/Check GGUF files and estimate the memory usage and maximum tokens per second.
  ☆215Updated 2 months ago
• sgl-project / ome
  OME is a Kubernetes operator for enterprise-grade management and serving of Large Language Models (LLMs)
  ☆307Updated last week
• InftyAI / Manta
  💫 A lightweight p2p-based cache system for model distributions on Kubernetes. Reframing now to make it an unified cache system with POSI…
  ☆24Updated 11 months ago
• NVIDIA / knavigator
  knavigator is a development, testing, and optimization toolkit for AI/ML scheduling systems at scale on Kubernetes.
  ☆71Updated 3 months ago
• microsoft / AIOpsLab
  A holistic framework to enable the design, development, and evaluation of autonomous AIOps agents.
  ☆729Updated this week
• NVIDIA / k8s-nim-operator
  An Operator for deployment and maintenance of NVIDIA NIMs and NeMo microservices in a Kubernetes environment.
  ☆134Updated this week
• llm-d / llm-d-kv-cache-manager
  Distributed KV cache coordinator
  ☆85Updated this week
• bentoml / BentoVLLM
  Self-host LLMs with vLLM and BentoML
  ☆156Updated 2 weeks ago
• kubernetes-sigs / agent-sandbox
  agent-sandbox enables easy management of isolated, stateful, singleton workloads, ideal for use cases like AI agent runtimes.
  ☆135Updated last week
• qingwave / kubewizard
  ✨Kubewizard is An AI-Agent for automated Kubernetes troubleshooting, and management, based on LangChain and k8s related tools.
  ☆27Updated 10 months ago
• fuzzylabs / sre-agent
  A Site Reliability Engineer AI agent that can monitor application and infrastructure logs, diagnose issues, and report on diagnostics.
  ☆108Updated last month
• run-ai / fake-gpu-operator
  ☆159Updated 3 weeks ago
• ai-dynamo / grove
  Kubernetes enhancements for Network Topology Aware Gang Scheduling & Autoscaling
  ☆86Updated last week
• tensorchord / openmodelz
  Autoscale LLM (vLLM, SGLang, LMDeploy) inferences on Kubernetes (and others)
  ☆276Updated 2 years ago
• vllm-project / vllm-project.github.io
  ☆23Updated 2 weeks ago
• Project-HAMi / volcano-vgpu-device-plugin
  Device-plugin for volcano vgpu which support hard resource isolation
  ☆128Updated last month
• gpustack / llama-box
  LM inference server implementation based on *.cpp.
  ☆289Updated 3 months ago
• strowk / mcp-k8s-go
  MCP server connecting to Kubernetes
  ☆355Updated this week
• llm-d / llm-d-inference-scheduler
  Inference scheduler for llm-d
  ☆103Updated this week
• tensorchord / ai-infra-landscape
  This is a landscape of the infrastructure that powers the generative AI ecosystem
  ☆149Updated last year