makllama / makllamaLinks
MaK(Mac+Kubernetes)llama - Running and orchestrating large language models (LLMs) on Kubernetes with macOS nodes.
☆41Updated last year
Alternatives and similar repositories for makllama
Users that are interested in makllama are comparing it to the libraries listed below
Sorting:
- ☸️ Easy, advanced inference platform for large language models on Kubernetes. 🌟 Star to support our work!☆219Updated last week
- 🎉 An awesome & curated list of best LLMOps tools.☆134Updated this week
- 🚢 Yet another operator for running large language models on Kubernetes with ease. Powered by Ollama! 🐫☆196Updated this week
- A diverse, simple, and secure all-in-one LLMOps platform☆107Updated 9 months ago
- This project is designed to simulate GPU information, making it easier to test scenarios where a GPU is not available.☆48Updated 4 months ago
- A toolkit for discovering cluster network topology.☆56Updated this week
- Extensible generative AI platform on Kubernetes with OpenAI-compatible APIs.☆86Updated this week
- NVIDIA vGPU Device Manager manages NVIDIA vGPU devices on top of Kubernetes☆136Updated this week
- ☆124Updated this week
- 🧯 Kubernetes coverage for fault awareness and recovery, works for any LLMOps, MLOps, AI workloads.☆30Updated last week
- This is a landscape of the infrastructure that powers the generative AI ecosystem☆147Updated 8 months ago
- GPUd automates monitoring, diagnostics, and issue identification for GPUs☆385Updated this week
- An Operator for deployment and maintenance of NVIDIA NIMs and NeMo microservices in a Kubernetes environment.☆115Updated last week
- The NVIDIA GPU driver container allows the provisioning of the NVIDIA driver through the use of containers.☆118Updated this week
- Autoscale LLM (vLLM, SGLang, LMDeploy) inferences on Kubernetes (and others)☆270Updated last year
- knavigator is a development, testing, and optimization toolkit for AI/ML scheduling systems at scale on Kubernetes.☆67Updated 2 months ago
- ☆53Updated last week
- An Envoy inspired, ultimate LLM-first gateway for LLM serving and downstream application developers and enterprises☆20Updated 2 months ago
- ☆228Updated last week
- ☆159Updated 3 weeks ago
- OME is a Kubernetes operator for enterprise-grade management and serving of Large Language Models (LLMs)☆156Updated this week
- Self-hosted huggingface mirror service. 自建huggingface镜像服务。☆174Updated last month
- Envoy AI Gateway is an open source project for using Envoy Gateway to handle request traffic from application clients to Generative AI se…☆340Updated this week
- MCP server connecting to Kubernetes☆315Updated this week
- Command-line tools for managing OCI model artifacts, which are bundled based on Model Spec☆30Updated this week
- Run Slurm on Kubernetes. A Slinky project.☆124Updated 2 weeks ago
- The main purpose of runtime copilot is to assist with node runtime management tasks such as configuring registries, upgrading versions, i…☆12Updated 2 years ago
- LeaderWorkerSet: An API for deploying a group of pods as a unit of replication☆514Updated this week
- A QA system based on k8s-specific knowledge build on ChatGLM2-6B, serving by Ray.☆10Updated last year
- Your AI Kubernetes Expert☆183Updated 2 years ago