tensorchord / ai-infra-landscape
This is a landscape of the infrastructure that powers the generative AI ecosystem
☆130Updated last month
Related projects ⓘ
Alternatives and complementary repositories for ai-infra-landscape
- Finetune LLMs on K8s by using Runbooks☆169Updated 2 months ago
- Pretrain, finetune and serve LLMs on Intel platforms with Ray☆103Updated last week
- One-click machine learning deployment (LLM, text-to-image and so on) at scale on any cluster (GCP, AWS, Lambda labs, your home lab, or ev…☆239Updated last year
- An Operator for deployment and maintenance of NVIDIA NIMs and NeMo microservices in a Kubernetes environment.☆57Updated this week
- 🎉 An awesome & curated list of best LLMOps tools.☆29Updated last month
- 📡 Deploy AI models and apps to Kubernetes without developing a hernia☆31Updated 5 months ago
- ChatData 🔍 📖 brings RAG to real applications with FREE✨ knowledge bases. Now enjoy your chat with 6 million wikipedia pages and 2 milli…☆155Updated last week
- Community-maintained Kubernetes config and Helm chart for Langfuse☆54Updated this week
- CUDA checkpoint and restore utility☆226Updated 7 months ago
- Tutorial for building LLM router☆163Updated 4 months ago
- Vector Database with support for late interaction and token level embeddings.☆54Updated last month
- Foyle is a copilot to help developers deploy and operate their applications.☆110Updated this week
- ☆120Updated this week
- Action library for AI Agent☆191Updated 2 weeks ago
- Helm charts for the KubeRay project☆33Updated last month
- ☆214Updated this week
- A simple service that integrates vLLM with Ray Serve for fast and scalable LLM serving.☆54Updated 7 months ago
- cluster/scheduler health monitoring for GPU jobs on k8s☆44Updated this week
- ☆144Updated 10 months ago
- Repository for open inference protocol specification☆42Updated 4 months ago
- A diverse, simple, and secure all-in-one LLMOps platform☆86Updated 2 months ago
- LeaderWorkerSet: An API for deploying a group of pods as a unit of replication☆140Updated this week
- Graphsignal Tracer for Python☆202Updated 3 months ago
- K8s device plugin for GPU sharing☆97Updated last year
- [deprecated] AI Gateway - core infrastructure stack for building production-ready AI Applications☆155Updated 7 months ago
- ☸️ Easy, advanced inference platform for large language models on Kubernetes. 🌟 Star to support our work!☆29Updated this week
- Self-host LLMs with vLLM and BentoML☆74Updated this week
- Python client library for improving your LLM app accuracy☆96Updated this week
- Tune efficiently any LLM model from HuggingFace using distributed training (multiple GPU) and DeepSpeed. Uses Ray AIR to orchestrate the …☆53Updated last year
- Open Weight, tool-calling LLMs☆149Updated 3 weeks ago