tensorchord / openmodelz
Autoscale LLM (vLLM, SGLang, LMDeploy) inferences on Kubernetes (and others)
☆253Updated last year
Alternatives and similar repositories for openmodelz:
Users that are interested in openmodelz are comparing it to the libraries listed below
- OpenAI compatible API for LLMs and embeddings (LLaMA, Vicuna, ChatGLM and many others)☆272Updated last year
- This is a landscape of the infrastructure that powers the generative AI ecosystem☆137Updated 4 months ago
- 🎉 An awesome & curated list of best LLMOps tools.☆41Updated 2 weeks ago
- a local implementation of OpenAI Assistants API: myla stands for MY Local Assistant☆53Updated 6 months ago
- EvalGPT is an code interpreter framework that utilizes large language models to automate the process of code-writing and execution, deliv…☆250Updated last year
- A diverse, simple, and secure all-in-one LLMOps platform☆100Updated 5 months ago
- Self-host LLMs with vLLM and BentoML☆90Updated this week
- Octogen is an Open-Source Code Interpreter Agent Framework☆254Updated 6 months ago
- Lepton Examples☆141Updated 2 months ago
- ChatData 🔍 📖 brings RAG to real applications with FREE✨ knowledge bases. Now enjoy your chat with 6 million wikipedia pages and 2 milli…☆167Updated 3 months ago
- A simple service that integrates vLLM with Ray Serve for fast and scalable LLM serving.☆63Updated 10 months ago
- An open-source cloud-native of large multi-modal models (LMMs) serving framework.☆161Updated last year
- Examples on how to use LangChain and Ray☆226Updated last year
- an MLOps/LLMOps platform☆225Updated 2 months ago
- Using LlamaIndex with Ray for productionizing LLM applications☆71Updated last year
- Cloud-native way to provide elastic Jupyter Notebooks on Kubernetes. Run remote kernels, natively.☆195Updated 2 years ago
- GPUd automates monitoring, diagnostics, and issue identification for GPUs☆281Updated this week
- ☆448Updated last year
- ☆53Updated 2 months ago
- Your AI Kubernetes Expert☆176Updated last year
- LLMPerf is a library for validating and benchmarking LLMs☆788Updated 2 months ago
- Turn any OCR models into online inference API endpoint 🚀 🌖☆53Updated 2 weeks ago
- A lightweight version of Milvus☆303Updated this week
- RayLLM - LLMs on Ray☆1,260Updated 9 months ago
- Open-source observability for your LLM application.☆49Updated 2 months ago
- Model Deployment at Scale on Kubernetes 🦄️☆795Updated 9 months ago
- Benchmarking suite for popular AI APIs☆81Updated 3 weeks ago
- Machine Learning Serving focused on GenAI with simplicity as the top priority.☆58Updated last month
- Finetune LLMs on K8s by using Runbooks☆170Updated 6 months ago
- ☆164Updated this week