tensorchord / ai-infra-landscapeLinks
This is a landscape of the infrastructure that powers the generative AI ecosystem
☆148Updated 10 months ago
Alternatives and similar repositories for ai-infra-landscape
Users that are interested in ai-infra-landscape are comparing it to the libraries listed below
Sorting:
- Autoscale LLM (vLLM, SGLang, LMDeploy) inferences on Kubernetes (and others)☆270Updated last year
- Finetune LLMs on K8s by using Runbooks☆169Updated 11 months ago
- 🎉 An awesome & curated list of best LLMOps tools.☆147Updated 2 weeks ago
- A diverse, simple, and secure all-in-one LLMOps platform☆107Updated 11 months ago
- knavigator is a development, testing, and optimization toolkit for AI/ML scheduling systems at scale on Kubernetes.☆69Updated last month
- Pretrain, finetune and serve LLMs on Intel platforms with Ray☆128Updated last week
- 📡 Deploy AI models and apps to Kubernetes without developing a hernia☆32Updated last year
- Repository for open inference protocol specification☆59Updated 3 months ago
- An Operator for deployment and maintenance of NVIDIA NIMs and NeMo microservices in a Kubernetes environment.☆124Updated last week
- GPUd automates monitoring, diagnostics, and issue identification for GPUs☆413Updated this week
- OME is a Kubernetes operator for enterprise-grade management and serving of Large Language Models (LLMs)☆226Updated last week
- High-scale LLM gateway, written in Rust. OpenTelemetry-based observability included☆123Updated last week
- Extensible generative AI platform on Kubernetes with OpenAI-compatible APIs.☆88Updated last week
- Knowledge for GPTScript☆29Updated 9 months ago
- Your AI Kubernetes Expert☆183Updated 2 years ago
- A distributed engine for elastic workload, e.g. AI Agent, Quant☆29Updated this week
- This repository contains statistics about the AI Infrastructure products.☆17Updated 5 months ago
- xet client tech, used in huggingface_hub☆171Updated this week
- Helm charts for the KubeRay project☆50Updated last month
- Holistic job manager on Kubernetes☆116Updated last year
- Open Weight, tool-calling LLMs☆155Updated 10 months ago
- ☆38Updated 2 weeks ago
- Helm charts to deploy Weaviate to k8s☆63Updated last week
- Embed machine learning models in your Dockerfile☆94Updated last week
- Custom Scheduler to deploy ML models to TRTIS for GPU Sharing☆12Updated 5 years ago
- ☆238Updated last week
- ☆63Updated 4 months ago
- ☸️ Easy, advanced inference platform for large language models on Kubernetes. 🌟 Star to support our work!☆239Updated last week
- A benchmarking tool for comparing different LLM API providers' DeepSeek model deployments.☆29Updated 4 months ago
- A simple service that integrates vLLM with Ray Serve for fast and scalable LLM serving.☆70Updated last year