tensorchord / openmodelzLinks
Autoscale LLM (vLLM, SGLang, LMDeploy) inferences on Kubernetes (and others)
☆275Updated last year
Alternatives and similar repositories for openmodelz
Users that are interested in openmodelz are comparing it to the libraries listed below
Sorting:
- OpenAI compatible API for LLMs and embeddings (LLaMA, Vicuna, ChatGLM and many others)☆275Updated last year
- This is a landscape of the infrastructure that powers the generative AI ecosystem☆149Updated 11 months ago
- EvalGPT is an code interpreter framework that utilizes large language models to automate the process of code-writing and execution, deliv…☆248Updated 2 years ago
- A diverse, simple, and secure all-in-one LLMOps platform☆108Updated last year
- an MLOps/LLMOps platform☆234Updated 9 months ago
- a local implementation of OpenAI Assistants API: myla stands for MY Local Assistant☆56Updated last year
- Turn any OCR models into online inference API endpoint 🚀 🌖☆56Updated 6 months ago
- A simple service that integrates vLLM with Ray Serve for fast and scalable LLM serving.☆72Updated last year
- Self-host LLMs with vLLM and BentoML☆150Updated this week
- An open-source cloud-native of large multi-modal models (LMMs) serving framework.☆167Updated 2 years ago
- Pretrain, finetune and serve LLMs on Intel platforms with Ray☆132Updated last week
- Open-source observability for your LLM application.☆53Updated 9 months ago
- Lepton Examples☆145Updated last month
- A lightweight version of Milvus☆373Updated 2 weeks ago
- This repository contains statistics about the AI Infrastructure products.☆17Updated 7 months ago
- Examples on how to use LangChain and Ray☆229Updated 2 years ago
- Turn PostgreSQL into your search engine in a Pythonic way.☆50Updated last month
- GPUd automates monitoring, diagnostics, and issue identification for GPUs☆436Updated last week
- Self-hosted huggingface mirror service. 自建huggingface镜像服务。☆195Updated 2 months ago
- ChatData 🔍 📖 brings RAG to real applications with FREE✨ knowledge bases. Now enjoy your chat with 6 million wikipedia pages and 2 milli…☆178Updated 10 months ago
- Octogen is an Open-Source Code Interpreter Agent Framework☆259Updated last year
- TiDB Vector SDK for Python, including code examples. Join our Discord: https://discord.gg/XzSW23Jg9p☆58Updated 2 months ago
- OpenAI compatible API for open source LLMs☆16Updated last year
- Cloud-native way to provide elastic Jupyter Notebooks on Kubernetes. Run remote kernels, natively.☆203Updated 3 years ago
- Run AI generated code in isolated sandboxes☆109Updated 7 months ago
- Using LlamaIndex, Redis, and OpenAI to chat with PDF documents. Supplementary material for blog post on Microsoft Developer Blog☆114Updated last year
- setup the env for vllm users☆16Updated last year
- Awesome multilingual OCR toolkits based on PaddlePaddle (practical ultra lightweight OCR system, support 80+ languages recognition, provi…☆39Updated 6 months ago
- Benchmarking suite for popular AI APIs☆87Updated 7 months ago
- parallel fetch☆138Updated 2 weeks ago