makllama / makllamaLinks
MaK(Mac+Kubernetes)llama - Running and orchestrating large language models (LLMs) on Kubernetes with macOS nodes.
☆43Updated last year
Alternatives and similar repositories for makllama
Users that are interested in makllama are comparing it to the libraries listed below
Sorting:
- ☸️ Easy, advanced inference platform for large language models on Kubernetes. 🌟 Star to support our work!☆278Updated last week
- A diverse, simple, and secure all-in-one LLMOps platform☆109Updated last year
- 🎉 An awesome & curated list of best LLMOps tools.☆176Updated 3 weeks ago
- An Operator for deployment and maintenance of NVIDIA NIMs and NeMo microservices in a Kubernetes environment.☆140Updated last week
- ☆179Updated last week
- 🚢 Yet another operator for running large language models on Kubernetes with ease. Powered by Ollama! 🐫☆226Updated this week
- agent-sandbox enables easy management of isolated, stateful, singleton workloads, ideal for use cases like AI agent runtimes.☆506Updated last week
- Command-line tools for managing OCI model artifacts, which are bundled based on Model Spec☆57Updated this week
- Extensible generative AI platform on Kubernetes with OpenAI-compatible APIs.☆91Updated 2 months ago
- An Envoy inspired, ultimate LLM-first gateway for LLM serving and downstream application developers and enterprises☆24Updated 8 months ago
- NVIDIA vGPU Device Manager manages NVIDIA vGPU devices on top of Kubernetes☆152Updated this week
- Kubernetes-native AI serving platform for scalable model serving.☆95Updated this week
- This project is designed to simulate GPU information, making it easier to test scenarios where a GPU is not available.☆59Updated 9 months ago
- knavigator is a development, testing, and optimization toolkit for AI/ML scheduling systems at scale on Kubernetes.☆72Updated 5 months ago
- Simplified Data Management and Sharing for Kubernetes☆14Updated last week
- Your AI Kubernetes Expert☆186Updated 2 years ago
- This is a landscape of the infrastructure that powers the generative AI ecosystem☆151Updated last year
- 🧯 Kubernetes coverage for fault awareness and recovery, works for any LLMOps, MLOps, AI workloads.☆33Updated last week
- Run native macOS workloads on Kubernetes☆336Updated 2 months ago
- ☆185Updated 3 weeks ago
- ✨Kubewizard is An AI-Agent for automated Kubernetes troubleshooting, and management, based on LangChain and k8s related tools.☆29Updated 11 months ago
- The main purpose of runtime copilot is to assist with node runtime management tasks such as configuring registries, upgrading versions, i…☆12Updated 2 years ago
- GPUd automates monitoring, diagnostics, and issue identification for GPUs☆466Updated this week
- Kubernetes APIServer 高性能代理组件，代理 APIServer 的 List 请求，其它类型的请求会直接反向代理到原生 APIServer。 CKube 还额外支持了分页、搜索和索引等功能。 并且，CKube 100% 兼容原生 kubectl 和 ku…☆19Updated 3 years ago
- llm-d helm charts and deployment examples☆48Updated last week
- Autoscale LLM (vLLM, SGLang, LMDeploy) inferences on Kubernetes (and others)☆278Updated 2 years ago
- Knowledge for GPTScript☆29Updated last year
- LeaderWorkerSet: An API for deploying a group of pods as a unit of replication☆639Updated this week
- Distributed KV cache coordinator☆92Updated this week
- GenAI inference performance benchmarking tool☆137Updated last week