InftyAI / Awesome-LLMOps

Related projects: ⓘ

• InftyAI / llmaz
  ☸️ Easy, advanced inference platform for large language models on Kubernetes
  ☆15Updated this week
• tensorchord / llmspec
  OpenAI compatible API for open source LLMs
  ☆15Updated 10 months ago
• asprenger / ray_vllm_inference
  A simple service that integrates vLLM with Ray Serve for fast and scalable LLM serving.
  ☆45Updated 5 months ago
• kerthcet / k8s-specific-knowledge-base
  A QA system based on k8s-specific knowledge build on ChatGLM2-6B, serving by Ray.
  ☆9Updated last year
• kubeagi / arcadia
  A diverse, simple, and secure one-stop LLMOps platform
  ☆78Updated 4 months ago
• gptscript-ai / knowledge
  Knowledge for GPTScript
  ☆24Updated this week
• jjoneson / k8s-langchain
  Kuberentes LangChain Agent - Interact with Kubernetes Clusters using LLMs
  ☆23Updated last year
• kubernetes-sigs / llm-instance-gateway
  LLM Instance gateway implementation.
  ☆18Updated last week
• kubernetes / dynamic-resource-allocation
  ☆25Updated this week
• foundation-model-stack / multi-nic-cni
  ☆33Updated 3 weeks ago
• tensorchord / ai-infra-statistics
  This repository contains statistics about the AI Infrastructure products.
  ☆18Updated 2 months ago
• unifyai / aibench-llm-endpoints
  Runner in charge of collecting metrics from LLM inference endpoints for the Unify Hub
  ☆16Updated 7 months ago
• substratusai / vllm-docker
  ☆43Updated 3 weeks ago
• makllama / makllama
  MaK(Mac+Kubernetes)llama
  ☆23Updated 3 months ago
• AliyunContainerService / spot-instance-advisor
  spot-instance-advisor is command line tool to get the cheapest group of spot instanceTypes.
  ☆21Updated 4 years ago
• copilot-io / runtime-copilot
  The main purpose of runtime copilot is to assist with node runtime management tasks such as configuring registries, upgrading versions, i…
  ☆12Updated last year
• bentoml / BentoVLLM
  Self-host LLMs with vLLM and BentoML
  ☆62Updated this week
• tensorchord / ai-infra-landscape
  This is a landscape of the infrastructure that powers the generative AI ecosystem
  ☆123Updated 2 weeks ago
• Proxati / llm_proxy
  AI aware proxy
  ☆16Updated this week
• substratusai / helm
  ☆15Updated last month
• kubevela / prism
  API Extensions for core KubeVela.
  ☆12Updated last week
• NVIDIA / knavigator
  knavigator is a development, testing, and optimization toolkit for AI/ML scheduling systems at scale on Kubernetes.
  ☆47Updated 3 weeks ago
• amogkam / llama_index_ray
  Using LlamaIndex with Ray for productionizing LLM applications
  ☆70Updated last year
• langchain-ai / helm
  Repository hosting Langchain helm charts.
  ☆38Updated last week
• leptonai / gpud
  ☆158Updated this week
• myscale / myscale-telemetry
  Open-source observability for your LLM application.
  ☆40Updated 2 months ago
• nicoloboschi / local-kubernetes-runner
  CLI for spins up a K8s cluster locally in 10 seconds.
  ☆16Updated 3 months ago
• run-llama / llamabot
  ☆116Updated 7 months ago
• NVIDIA / cloud-native-docs
  Documentation repository for NVIDIA Cloud Native Technologies
  ☆14Updated this week
• tensorchord / openmodelz
  One-click machine learning deployment (LLM, text-to-image and so on) at scale on any cluster (GCP, AWS, Lambda labs, your home lab, or ev…
  ☆239Updated 10 months ago