tensorchord / ai-infra-landscapeLinks
This is a landscape of the infrastructure that powers the generative AI ecosystem
β148Updated 9 months ago
Alternatives and similar repositories for ai-infra-landscape
Users that are interested in ai-infra-landscape are comparing it to the libraries listed below
Sorting:
- Autoscale LLM (vLLM, SGLang, LMDeploy) inferences on Kubernetes (and others)β270Updated last year
- π An awesome & curated list of best LLMOps tools.β141Updated this week
- Finetune LLMs on K8s by using Runbooksβ169Updated 11 months ago
- OME is a Kubernetes operator for enterprise-grade management and serving of Large Language Models (LLMs)β202Updated this week
- Pretrain, finetune and serve LLMs on Intel platforms with Rayβ128Updated 3 weeks ago
- A diverse, simple, and secure all-in-one LLMOps platformβ107Updated 10 months ago
- β231Updated this week
- Holistic job manager on Kubernetesβ117Updated last year
- Repository for open inference protocol specificationβ59Updated 2 months ago
- An Operator for deployment and maintenance of NVIDIA NIMs and NeMo microservices in a Kubernetes environment.β119Updated this week
- Extensible generative AI platform on Kubernetes with OpenAI-compatible APIs.β87Updated this week
- π‘ Deploy AI models and apps to Kubernetes without developing a herniaβ32Updated last year
- GPUd automates monitoring, diagnostics, and issue identification for GPUsβ405Updated this week
- High-scale LLM gateway, written in Rust. OpenTelemetry-based observability includedβ116Updated 2 weeks ago
- knavigator is a development, testing, and optimization toolkit for AI/ML scheduling systems at scale on Kubernetes.β69Updated 2 weeks ago
- β38Updated this week
- Open Weight, tool-calling LLMsβ154Updated 9 months ago
- Kubernetes Operator, ansible playbooks, and production scripts for large-scale AIStore deployments on Kubernetes.β106Updated this week
- A toolkit for discovering cluster network topology.β59Updated last week
- A benchmarking tool for comparing different LLM API providers' DeepSeek model deployments.β29Updated 4 months ago
- βΈοΈ Easy, advanced inference platform for large language models on Kubernetes. π Star to support our work!β228Updated last week
- Helm charts for the KubeRay projectβ50Updated 2 weeks ago
- Helm charts for llm-dβ51Updated last week
- JobSet: a k8s native API for distributed ML training and HPC workloadsβ246Updated this week
- Self-host LLMs with vLLM and BentoMLβ138Updated last week
- A distributed engine for elastic workloadβ27Updated this week
- πͺΆ Lightweight OpenAI drop-in replacement for Kubernetesβ146Updated last year
- Embed machine learning models in your Dockerfileβ93Updated last week
- Modular, open source LLMOps stack that separates concerns: LiteLLM unifies LLM APIs, manages routing and cost controls, and ensures high-β¦β107Updated 5 months ago
- Knowledge for GPTScriptβ29Updated 9 months ago