makllama / makllamaLinks
MaK(Mac+Kubernetes)llama - Running and orchestrating large language models (LLMs) on Kubernetes with macOS nodes.
☆42Updated last year
Alternatives and similar repositories for makllama
Users that are interested in makllama are comparing it to the libraries listed below
Sorting:
- A diverse, simple, and secure all-in-one LLMOps platform☆108Updated 11 months ago
- ☸️ Easy, advanced inference platform for large language models on Kubernetes. 🌟 Star to support our work!☆252Updated this week
- Command-line tools for managing OCI model artifacts, which are bundled based on Model Spec☆38Updated this week
- 🎉 An awesome & curated list of best LLMOps tools.☆154Updated this week
- 🚢 Yet another operator for running large language models on Kubernetes with ease. Powered by Ollama! 🐫☆213Updated last week
- An Envoy inspired, ultimate LLM-first gateway for LLM serving and downstream application developers and enterprises☆23Updated 4 months ago
- knavigator is a development, testing, and optimization toolkit for AI/ML scheduling systems at scale on Kubernetes.☆69Updated last month
- A toolkit for discovering cluster network topology.☆67Updated this week
- This project is designed to simulate GPU information, making it easier to test scenarios where a GPU is not available.☆51Updated 6 months ago
- ☆144Updated last week
- The main purpose of runtime copilot is to assist with node runtime management tasks such as configuring registries, upgrading versions, i…☆12Updated 2 years ago
- The NVIDIA Driver Manager is a Kubernetes component which assist in seamless upgrades of NVIDIA Driver on each node of the cluster.☆37Updated this week
- An Operator for deployment and maintenance of NVIDIA NIMs and NeMo microservices in a Kubernetes environment.☆125Updated this week
- NVIDIA vGPU Device Manager manages NVIDIA vGPU devices on top of Kubernetes☆141Updated this week
- Extensible generative AI platform on Kubernetes with OpenAI-compatible APIs.☆89Updated this week
- ✨Kubewizard is An AI-Agent for automated Kubernetes troubleshooting, and management, based on LangChain and k8s related tools.☆25Updated 8 months ago
- Kubernetes APIServer 高性能代理组件，代理 APIServer 的 List 请求，其它类型的请求会直接反向代理到原生 APIServer。 CKube 还额外支持了分页、搜索和索引等功能。 并且，CKube 100% 兼容原生 kubectl 和 ku…☆19Updated 2 years ago
- GPUd automates monitoring, diagnostics, and issue identification for GPUs☆425Updated this week
- This is a landscape of the infrastructure that powers the generative AI ecosystem☆149Updated 11 months ago
- 🧯 Kubernetes coverage for fault awareness and recovery, works for any LLMOps, MLOps, AI workloads.☆33Updated this week
- 💫 A lightweight p2p-based cache system for model distributions on Kubernetes. Reframing now to make it an unified cache system with POSI…☆24Updated 9 months ago
- Holistic job manager on Kubernetes☆116Updated last year
- ☆39Updated this week
- Run native macOS workloads on Kubernetes☆310Updated 4 months ago
- Self-hosted huggingface mirror service. 自建huggingface镜像服务。☆190Updated last month
- Autoscale LLM (vLLM, SGLang, LMDeploy) inferences on Kubernetes (and others)☆273Updated last year
- Inference scheduler for llm-d☆87Updated this week
- ☆168Updated last week
- Distributed KV cache coordinator☆68Updated this week
- MCP server connecting to Kubernetes☆345Updated last week