makllama / makllamaLinks
MaK(Mac+Kubernetes)llama - Running and orchestrating large language models (LLMs) on Kubernetes with macOS nodes.
☆43Updated last year
Alternatives and similar repositories for makllama
Users that are interested in makllama are comparing it to the libraries listed below
Sorting:
- ☸️ Easy, advanced inference platform for large language models on Kubernetes. 🌟 Star to support our work!☆270Updated last week
- A diverse, simple, and secure all-in-one LLMOps platform☆109Updated last year
- 🎉 An awesome & curated list of best LLMOps tools.☆172Updated last week
- An Envoy inspired, ultimate LLM-first gateway for LLM serving and downstream application developers and enterprises☆24Updated 7 months ago
- knavigator is a development, testing, and optimization toolkit for AI/ML scheduling systems at scale on Kubernetes.☆72Updated 4 months ago
- This is a landscape of the infrastructure that powers the generative AI ecosystem☆151Updated last year
- Command-line tools for managing OCI model artifacts, which are bundled based on Model Spec☆50Updated this week
- NVIDIA vGPU Device Manager manages NVIDIA vGPU devices on top of Kubernetes☆152Updated this week
- Run native macOS workloads on Kubernetes☆332Updated last month
- An Operator for deployment and maintenance of NVIDIA NIMs and NeMo microservices in a Kubernetes environment.☆140Updated this week
- 🚢 Yet another operator for running large language models on Kubernetes with ease. Powered by Ollama! 🐫☆223Updated this week
- ☆79Updated this week
- GPUd automates monitoring, diagnostics, and issue identification for GPUs☆456Updated this week
- Your AI Kubernetes Expert☆186Updated 2 years ago
- This project is designed to simulate GPU information, making it easier to test scenarios where a GPU is not available.☆56Updated 9 months ago
- The main purpose of runtime copilot is to assist with node runtime management tasks such as configuring registries, upgrading versions, i…☆12Updated 2 years ago
- agent-sandbox enables easy management of isolated, stateful, singleton workloads, ideal for use cases like AI agent runtimes.☆397Updated last week
- Open Weight, tool-calling LLMs☆155Updated last year
- Kubernetes APIServer 高性能代理组件，代理 APIServer 的 List 请求，其它类型的请求会直接反向代理到原生 APIServer。 CKube 还额外支持了分页、搜索和索引等功能。 并且，CKube 100% 兼容原生 kubectl 和 ku…☆19Updated 3 years ago
- Extensible generative AI platform on Kubernetes with OpenAI-compatible APIs.☆90Updated last month
- Autoscale LLM (vLLM, SGLang, LMDeploy) inferences on Kubernetes (and others)☆278Updated 2 years ago
- Knowledge for GPTScript☆29Updated last year
- A stress testing tool for the scheduler in a large-scale scenario.☆16Updated last year
- 🧯 Kubernetes coverage for fault awareness and recovery, works for any LLMOps, MLOps, AI workloads.☆33Updated this week
- ☆168Updated last month
- 💫 A lightweight p2p-based cache system for model distributions on Kubernetes. Reframing now to make it an unified cache system with POSI…☆24Updated last year
- A QA system based on k8s-specific knowledge build on ChatGLM2-6B, serving by Ray.☆10Updated 2 years ago
- Distributed KV cache coordinator☆91Updated this week
- MCP server connecting to Kubernetes☆365Updated last week
- Run Slurm on Kubernetes. A Slinky project.☆198Updated this week