tensorchord / ai-infra-landscapeLinks
This is a landscape of the infrastructure that powers the generative AI ecosystem
☆149Updated 11 months ago
Alternatives and similar repositories for ai-infra-landscape
Users that are interested in ai-infra-landscape are comparing it to the libraries listed below
Sorting:
- Autoscale LLM (vLLM, SGLang, LMDeploy) inferences on Kubernetes (and others)☆275Updated last year
- Finetune LLMs on K8s by using Runbooks☆170Updated last year
- A diverse, simple, and secure all-in-one LLMOps platform☆108Updated last year
- 🎉 An awesome & curated list of best LLMOps tools.☆161Updated last week
- Pretrain, finetune and serve LLMs on Intel platforms with Ray☆132Updated last week
- GPUd automates monitoring, diagnostics, and issue identification for GPUs☆436Updated last week
- knavigator is a development, testing, and optimization toolkit for AI/ML scheduling systems at scale on Kubernetes.☆69Updated 2 months ago
- This repository contains statistics about the AI Infrastructure products.☆17Updated 7 months ago
- An Operator for deployment and maintenance of NVIDIA NIMs and NeMo microservices in a Kubernetes environment.☆129Updated this week
- OME is a Kubernetes operator for enterprise-grade management and serving of Large Language Models (LLMs)☆279Updated last week
- xet client tech, used in huggingface_hub☆278Updated this week
- A benchmarking tool for comparing different LLM API providers' DeepSeek model deployments.☆29Updated 6 months ago
- Holistic job manager on Kubernetes☆116Updated last year
- Embed machine learning models in your Dockerfile☆95Updated 2 weeks ago
- Helm charts for the KubeRay project☆51Updated 2 months ago
- A toolkit for discovering cluster network topology.☆70Updated last week
- ☸️ Easy, advanced inference platform for large language models on Kubernetes. 🌟 Star to support our work!☆261Updated 2 weeks ago
- ☆40Updated last week
- Self-host LLMs with vLLM and BentoML☆150Updated this week
- ☆255Updated this week
- 📡 Deploy AI models and apps to Kubernetes without developing a hernia☆33Updated last year
- ☆64Updated 6 months ago
- Rust crates for XetHub☆67Updated 11 months ago
- ☆170Updated 3 weeks ago
- Repository for open inference protocol specification☆59Updated 4 months ago
- Extensible generative AI platform on Kubernetes with OpenAI-compatible APIs.☆91Updated last week
- agent-sandbox enables easy management of isolated, stateful, singleton workloads, ideal for use cases like AI agent runtimes.☆84Updated last week
- Helm charts for llm-d☆50Updated 2 months ago
- A simple service that integrates vLLM with Ray Serve for fast and scalable LLM serving.☆72Updated last year
- Your AI Kubernetes Expert☆186Updated 2 years ago