tensorchord / ai-infra-landscapeLinks
This is a landscape of the infrastructure that powers the generative AI ecosystem
☆146Updated 7 months ago
Alternatives and similar repositories for ai-infra-landscape
Users that are interested in ai-infra-landscape are comparing it to the libraries listed below
Sorting:
- 🎉 An awesome & curated list of best LLMOps tools.☆109Updated this week
- GPUd automates monitoring, diagnostics, and issue identification for GPUs☆362Updated this week
- A diverse, simple, and secure all-in-one LLMOps platform☆105Updated 8 months ago
- 🧯 Kubernetes coverage for fault awareness and recovery, works for any LLMOps, MLOps, AI workloads.☆30Updated 5 months ago
- Repository for open inference protocol specification☆56Updated 3 weeks ago
- Pretrain, finetune and serve LLMs on Intel platforms with Ray☆127Updated last month
- Helm charts for the KubeRay project☆43Updated last month
- A toolkit for discovering cluster network topology.☆53Updated this week
- Finetune LLMs on K8s by using Runbooks☆170Updated 9 months ago
- Envoy AI Gateway is an open source project for using Envoy Gateway to handle request traffic from application clients to Generative AI se…☆275Updated this week
- knavigator is a development, testing, and optimization toolkit for AI/ML scheduling systems at scale on Kubernetes.☆67Updated 3 weeks ago
- Autoscale LLM (vLLM, SGLang, LMDeploy) inferences on Kubernetes (and others)☆267Updated last year
- ☸️ Easy, advanced inference platform for large language models on Kubernetes. 🌟 Star to support our work!☆178Updated last week
- A simple service that integrates vLLM with Ray Serve for fast and scalable LLM serving.☆67Updated last year
- An Operator for deployment and maintenance of NVIDIA NIMs and NeMo microservices in a Kubernetes environment.☆107Updated this week
- ☆215Updated this week
- Helm charts for llm-d☆35Updated this week
- 📡 Deploy AI models and apps to Kubernetes without developing a hernia☆32Updated last year
- Extensible generative AI platform on Kubernetes with OpenAI-compatible APIs.☆79Updated this week
- A distributed engine for intelligent workload☆26Updated 3 months ago
- KAI Scheduler is an open source Kubernetes Native scheduler for AI workloads at large scale☆590Updated this week
- Runner in charge of collecting metrics from LLM inference endpoints for the Unify Hub☆17Updated last year
- Gateway API Inference Extension☆317Updated this week
- A collection of YAML files, Helm Charts, Operator code, and guides to act as an example reference implementation for NVIDIA NIM deploymen…☆182Updated this week
- GenAI inference performance benchmarking tool☆45Updated this week
- Tune efficiently any LLM model from HuggingFace using distributed training (multiple GPU) and DeepSpeed. Uses Ray AIR to orchestrate the …☆58Updated last year
- Holistic job manager on Kubernetes☆115Updated last year
- ChatData 🔍 📖 brings RAG to real applications with FREE✨ knowledge bases. Now enjoy your chat with 6 million wikipedia pages and 2 milli…☆172Updated 6 months ago
- CUDA checkpoint and restore utility☆341Updated 4 months ago
- Dynolog is a telemetry daemon for performance monitoring and tracing. It exports metrics from different components in the system like the…☆314Updated this week