Alternatives and similar repositories for llmos

Users that are interested in llmos are comparing it to the libraries listed below

• InftyAI / Awesome-LLMOps
  🎉 An awesome & curated list of best LLMOps tools.
  ☆165Updated last week
• kubeagi / arcadia
  A diverse, simple, and secure all-in-one LLMOps platform
  ☆108Updated last year
• InftyAI / llmaz
  ☸️ Easy, advanced inference platform for large language models on Kubernetes. 🌟 Star to support our work!
  ☆261Updated last week
• gpustack / gguf-parser-go
  Review/Check GGUF files and estimate the memory usage and maximum tokens per second.
  ☆211Updated 2 months ago
• qingwave / kubewizard
  ✨Kubewizard is An AI-Agent for automated Kubernetes troubleshooting, and management, based on LangChain and k8s related tools.
  ☆27Updated 9 months ago
• gpustack / llama-box
  LM inference server implementation based on *.cpp.
  ☆286Updated 2 months ago
• microsoft / AIOpsLab
  A holistic framework to enable the design, development, and evaluation of autonomous AIOps agents.
  ☆712Updated last week
• NVIDIA-AI-Blueprints / llm-router
  Route LLM requests to the best model for the task at hand.
  ☆113Updated last month
• nekomeowww / ollama-operator
  🚢 Yet another operator for running large language models on Kubernetes with ease. Powered by Ollama! 🐫
  ☆219Updated last week
• llmariner / llmariner
  Extensible generative AI platform on Kubernetes with OpenAI-compatible APIs.
  ☆90Updated 2 weeks ago
• substratusai / sandboxai
  Run AI generated code in isolated sandboxes
  ☆112Updated 8 months ago
• docker / model-runner
  Docker Model Runner
  ☆192Updated this week
• opea-project / GenAIInfra
  Containerization and cloud native suite for OPEA
  ☆70Updated 2 weeks ago
• kaito-project / aikit
  🏗️ Fine-tune, build, and deploy open-source LLMs easily!
  ☆483Updated this week
• Project-HAMi / volcano-vgpu-device-plugin
  Device-plugin for volcano vgpu which support hard resource isolation
  ☆113Updated last month
• clearml / clearml-fractional-gpu
  ClearML Fractional GPU - Run multiple containers on the same GPU with driver level memory limitation ✨ and compute time-slicing
  ☆81Updated last year
• run-ai / fake-gpu-operator
  ☆152Updated this week
• modelpack / modctl
  Command-line tools for managing OCI model artifacts, which are bundled based on Model Spec
  ☆45Updated this week
• llm-d / llm-d-inference-scheduler
  Inference scheduler for llm-d
  ☆99Updated last week
• NVIDIA / KAI-Scheduler
  KAI Scheduler is an open source Kubernetes Native scheduler for AI workloads at large scale
  ☆869Updated this week
• seal-io / appilot
  Chat to deploy and manage applications on any infrastructure
  ☆126Updated last year
• obot-platform / obot
  Open-source MCP Gateway and AI Platform
  ☆413Updated last week
• NVIDIA / knavigator
  knavigator is a development, testing, and optimization toolkit for AI/ML scheduling systems at scale on Kubernetes.
  ☆70Updated 3 months ago
• strowk / mcp-k8s-go
  MCP server connecting to Kubernetes
  ☆353Updated last month
• llm-d / llm-d-kv-cache-manager
  Distributed KV cache coordinator
  ☆79Updated last week
• fuzzylabs / sre-agent
  A Site Reliability Engineer AI agent that can monitor application and infrastructure logs, diagnose issues, and report on diagnostics.
  ☆104Updated 3 weeks ago
• bentoml / BentoVLLM
  Self-host LLMs with vLLM and BentoML
  ☆152Updated 2 weeks ago
• NVIDIA / vgpu-device-manager
  NVIDIA vGPU Device Manager manages NVIDIA vGPU devices on top of Kubernetes
  ☆144Updated this week
• NVIDIA / k8s-nim-operator
  An Operator for deployment and maintenance of NVIDIA NIMs and NeMo microservices in a Kubernetes environment.
  ☆130Updated this week
• wenhuwang / mcp-k8s-eye
  MCP Server for kubernetes management and diagnose your cluster and applications
  ☆25Updated 5 months ago