InftyAI / Awesome-LLMOps
π An awesome & curated list of best LLMOps tools.
β40Updated last week
Alternatives and similar repositories for Awesome-LLMOps:
Users that are interested in Awesome-LLMOps are comparing it to the libraries listed below
- βΈοΈ Easy, advanced inference platform for large language models on Kubernetes. π Star to support our work!β69Updated this week
- Self-host LLMs with vLLM and BentoMLβ86Updated this week
- OpenAI compatible API for open source LLMsβ15Updated last year
- Knowledge for GPTScriptβ29Updated 3 months ago
- A diverse, simple, and secure all-in-one LLMOps platformβ98Updated 4 months ago
- π‘ Deploy AI models and apps to Kubernetes without developing a herniaβ32Updated 8 months ago
- π« A lightweight p2p-based cache system for model distributions on Kubernetes. Reframing now to make it an unified cache system with POSIβ¦β20Updated 2 months ago
- Extensible generative AI platform on Kubernetes with OpenAI-compatible APIs.β48Updated this week
- A simple service that integrates vLLM with Ray Serve for fast and scalable LLM serving.β62Updated 10 months ago
- Document parser for RAGβ20Updated 3 months ago
- Open-source observability for your LLM application.β48Updated last month
- π§― Kubernetes coverage for fault awareness and recovery, works for any LLMOps, MLOps, AI workloads.β26Updated last month
- This repository contains statistics about the AI Infrastructure products.β18Updated 3 weeks ago
- This is a landscape of the infrastructure that powers the generative AI ecosystemβ135Updated 4 months ago
- AI aware proxyβ18Updated 5 months ago
- A Next.js version of Claude Aritfacts , inspired by llamacoderβ19Updated 4 months ago
- MaK(Mac+Kubernetes)llama - Running and orchestrating large language models (LLMs) on Kubernetes with macOS nodes.β35Updated 8 months ago
- A QA system based on k8s-specific knowledge build on ChatGLM2-6B, serving by Ray.β9Updated last year
- β19Updated 3 weeks ago
- Autoscale LLM (vLLM, SGLang, LMDeploy) inferences on Kubernetes (and others)β251Updated last year
- β53Updated last month
- Runner in charge of collecting metrics from LLM inference endpoints for the Unify Hubβ17Updated last year
- Yet another operator for running large language models on Kubernetes with ease. Powered by Ollama! π«β138Updated 2 weeks ago
- β12Updated last year
- Probably one of the lightest native RAG + Agent apps out thereοΌexperience the power of Agent-powered models and Agent-driven knowledge baβ¦β22Updated this week
- Self-host llmapi server, make it really easy for accessing LLMs !β37Updated last year
- β159Updated this week
- Kuberentes LangChain Agent - Interact with Kubernetes Clusters using LLMsβ23Updated last year
- A distributed engine for intelligent workloadβ23Updated last week