InftyAI / llmaz
☸️ Easy, advanced inference platform for large language models on Kubernetes. 🌟 Star to support our work!
☆79Updated this week
Alternatives and similar repositories for llmaz:
Users that are interested in llmaz are comparing it to the libraries listed below
- 💫 A lightweight p2p-based cache system for model distributions on Kubernetes. Reframing now to make it an unified cache system with POSI…☆20Updated 2 months ago
- Device-plugin for volcano vgpu which support hard resource isolation☆61Updated this week
- 🧯 Kubernetes coverage for fault awareness and recovery, works for any LLMOps, MLOps, AI workloads.☆26Updated 2 months ago
- knavigator is a development, testing, and optimization toolkit for AI/ML scheduling systems at scale on Kubernetes.☆62Updated last month
- Large language model fine-tuning capabilities based on cloud native and distributed computing.☆91Updated last year
- LeaderWorkerSet: An API for deploying a group of pods as a unit of replication☆296Updated this week
- A federation scheduler for multi-cluster☆30Updated last week
- Gateway API Inference Extension☆165Updated this week
- Federated middleware based on Karmada☆48Updated last year
- Example DRA driver that developers can fork and modify to get them started writing their own.☆61Updated this week
- JobSet: a k8s native API for distributed ML training and HPC workloads☆192Updated this week
- ControllerMesh is a solution that helps developers manage their controllers/operators better with enhanced isolation.☆63Updated last year
- ☆113Updated 2 years ago
- The API (CRD) of Volcano☆37Updated 2 weeks ago
- ☆12Updated last week
- elastic-gpu-scheduler is a Kubernetes scheduler extender for GPU resources scheduling.☆140Updated 2 years ago
- A Cloud-Native Service Catalog and Full Lifecycle Management Platform accross Multi-cloud and Edge☆33Updated last year
- Device plugins for Volcano, e.g. GPU☆115Updated 5 months ago
- ☆93Updated last month
- Fast is a Kubernetes CNI based on eBPF implementation☆35Updated last year
- Kubernetes APIServer 高性能代理组件,代理 APIServer 的 List 请求,其它类型的请求会直接反向代理到原生 APIServer。 CKube 还额外支持了分页、搜索和索引等功能。 并且,CKube 100% 兼容原生 kubectl 和 ku…☆19Updated 2 years ago
- Using CRDs to manage GPU resources in Kubernetes.☆197Updated 2 years ago
- [Moved to https://github.com/kubernetes-sigs/kwok] This is a fake kubelet. that can simulate any number of nodes and maintain pods on tho…☆63Updated 2 years ago
- katalyst aims to provide a universal solution to help improve resource utilization and optimize the overall costs in the cloud. This repo…☆41Updated 2 months ago
- Multi-cluster api gateway based on apiserver-aggregation.☆97Updated 2 months ago
- The main purpose of runtime copilot is to assist with node runtime management tasks such as configuring registries, upgrading versions, i…☆12Updated last year
- Fleetboard establishes an independent and unified parallel network, facilitating cross-cluster service discovery even in cases of IP over…☆26Updated 2 weeks ago
- HAMi-core compiles libvgpu.so, which ensures hard limit on GPU in container☆133Updated this week
- ☆15Updated 5 months ago
- elastic-gpu-agent is a Kubernetes device plugin for GPU resources allocation on node.☆54Updated 2 years ago