Alternatives and similar repositories for llmos

Users that are interested in llmos are comparing it to the libraries listed below

• kubeagi / arcadia
  A diverse, simple, and secure all-in-one LLMOps platform
  ☆109Updated last year
• InftyAI / llmaz
  ☸️ Easy, advanced inference platform for large language models on Kubernetes. 🌟 Star to support our work!
  ☆286Updated last week
• InftyAI / Awesome-LLMOps
  🎉 An awesome & curated list of best LLMOps tools.
  ☆184Updated last week
• makllama / makllama
  MaK(Mac+Kubernetes)llama - Running and orchestrating large language models (LLMs) on Kubernetes with macOS nodes.
  ☆43Updated last year
• NVIDIA-AI-Blueprints / llm-router
  Route LLM requests to the best model for the task at hand.
  ☆166Updated last week
• kubernetes-sigs / agent-sandbox
  agent-sandbox enables easy management of isolated, stateful, singleton workloads, ideal for use cases like AI agent runtimes.
  ☆702Updated last week
• strowk / mcp-k8s-go
  MCP server connecting to Kubernetes
  ☆368Updated 3 weeks ago
• clearml / clearml-fractional-gpu
  ClearML Fractional GPU - Run multiple containers on the same GPU with driver level memory limitation ✨ and compute time-slicing
  ☆88Updated 2 months ago
• gpustack / gguf-parser-go
  Review/Check GGUF files and estimate the memory usage and maximum tokens per second.
  ☆232Updated 2 weeks ago
• ai-dynamo / grove
  Kubernetes enhancements for Network Topology Aware Gang Scheduling & Autoscaling
  ☆143Updated this week
• gpustack / llama-box
  LM inference server implementation based on *.cpp.
  ☆294Updated last month
• inferx-net / inferx
  InferX: Inference as a Service Platform
  ☆146Updated this week
• llm-d / llm-d-kv-cache
  Distributed KV cache scheduling & offloading libraries
  ☆98Updated this week
• run-ai / fake-gpu-operator
  ☆195Updated this week
• volcano-sh / kthena
  Kubernetes-native AI serving platform for scalable model serving.
  ☆168Updated this week
• substratusai / sandboxai
  Run AI generated code in isolated sandboxes
  ☆137Updated 11 months ago
• qingwave / kubewizard
  ✨Kubewizard is An AI-Agent for automated Kubernetes troubleshooting, and management, based on LangChain and k8s related tools.
  ☆30Updated last year
• obot-platform / obot
  Complete MCP Platform -- Hosting, Registry, Gateway, and Chat Client
  ☆562Updated this week
• kaito-project / aikit
  🏗️ Fine-tune, build, and deploy open-source LLMs easily!
  ☆503Updated last week
• NVIDIA / topograph
  A toolkit for discovering cluster network topology.
  ☆90Updated this week
• llm-d / llm-d-inference-scheduler
  Inference scheduler for llm-d
  ☆120Updated this week
• llmariner / llmariner
  Extensible generative AI platform on Kubernetes with OpenAI-compatible APIs.
  ☆92Updated 3 months ago
• NVIDIA / vgpu-device-manager
  NVIDIA vGPU Device Manager manages NVIDIA vGPU devices on top of Kubernetes
  ☆154Updated this week
• micytao / vllm-playground
  A modern web interface for managing and interacting with vLLM servers (www.github.com/vllm-project/vllm). Supports both GPU and CPU modes…
  ☆331Updated this week
• Project-HAMi / volcano-vgpu-device-plugin
  Device-plugin for volcano vgpu which support hard resource isolation
  ☆142Updated 3 weeks ago
• NVIDIA / go-gpuallocator
  Go Abstraction for Allocating NVIDIA GPUs with Custom Policies
  ☆120Updated last month
• run-ai / runai-model-streamer
  ☆274Updated this week
• InftyAI / Manta
  💫 A lightweight p2p-based cache system for model distributions on Kubernetes. Reframing now to make it an unified cache system with POSI…
  ☆25Updated last year
• tensorchord / ai-infra-landscape
  This is a landscape of the infrastructure that powers the generative AI ecosystem
  ☆152Updated last year
• sgl-project / ome
  Open Model Engine (OME) — Kubernetes operator for LLM serving, GPU scheduling, and model lifecycle management. Works with SGLang, vLLM, T…
  ☆356Updated this week