tensorchord / openmodelzLinks
Autoscale LLM (vLLM, SGLang, LMDeploy) inferences on Kubernetes (and others)
☆267Updated last year
Alternatives and similar repositories for openmodelz
Users that are interested in openmodelz are comparing it to the libraries listed below
Sorting:
- OpenAI compatible API for LLMs and embeddings (LLaMA, Vicuna, ChatGLM and many others)☆275Updated last year
- EvalGPT is an code interpreter framework that utilizes large language models to automate the process of code-writing and execution, deliv…☆250Updated last year
- A simple service that integrates vLLM with Ray Serve for fast and scalable LLM serving.☆67Updated last year
- Self-host LLMs with vLLM and BentoML☆114Updated this week
- An open-source cloud-native of large multi-modal models (LMMs) serving framework.☆163Updated last year
- Turn any OCR models into online inference API endpoint 🚀 🌖☆55Updated 2 months ago
- Examples on how to use LangChain and Ray☆230Updated last year
- A diverse, simple, and secure all-in-one LLMOps platform☆105Updated 8 months ago
- This is a landscape of the infrastructure that powers the generative AI ecosystem☆146Updated 7 months ago
- ChatData 🔍 📖 brings RAG to real applications with FREE✨ knowledge bases. Now enjoy your chat with 6 million wikipedia pages and 2 milli…☆172Updated 6 months ago
- GPUd automates monitoring, diagnostics, and issue identification for GPUs☆362Updated this week
- Pretrain, finetune and serve LLMs on Intel platforms with Ray☆127Updated last month
- a local implementation of OpenAI Assistants API: myla stands for MY Local Assistant☆54Updated 8 months ago
- Machine Learning Serving focused on GenAI with simplicity as the top priority.☆57Updated last month
- RayLLM - LLMs on Ray (Archived). Read README for more info.☆1,260Updated 2 months ago
- A prompting library☆165Updated 8 months ago
- parallel fetch☆127Updated 2 weeks ago
- Open-source observability for your LLM application.☆50Updated 5 months ago
- Using LlamaIndex with Ray for productionizing LLM applications☆71Updated last year
- Lepton Examples☆141Updated last week
- A high-performance inference system for large language models, designed for production environments.☆441Updated this week
- Cloud-native way to provide elastic Jupyter Notebooks on Kubernetes. Run remote kernels, natively.☆197Updated 3 years ago
- ☆459Updated last year
- Inferflow is an efficient and highly configurable inference engine for large language models (LLMs).☆242Updated last year
- ☆214Updated this week
- ☆52Updated 6 months ago
- GenAIOps on Kubernetes: A collection of reference architectures for running GenAI at scale on Kubernetes using OSS tooling☆130Updated 7 months ago
- ☆260Updated 2 weeks ago
- ☆53Updated 8 months ago
- Using LlamaIndex, Redis, and OpenAI to chat with PDF documents. Supplementary material for blog post on Microsoft Developer Blog☆113Updated last year