bentoml / BentoVLLMLinks
Self-host LLMs with vLLM and BentoML
☆114Updated this week
Alternatives and similar repositories for BentoVLLM
Users that are interested in BentoVLLM are comparing it to the libraries listed below
Sorting:
- Tutorial for building LLM router☆207Updated 10 months ago
- ☆60Updated 2 months ago
- Evaluate and Enhance Your LLM Deployments for Real-World Inference Needs☆317Updated this week
- Fine-tune an LLM to perform batch inference and online serving.☆111Updated 3 weeks ago
- ☆101Updated 9 months ago
- Using LlamaIndex with Ray for productionizing LLM applications☆71Updated last year
- Ready-to-go containerized RAG service. Implemented with text-embedding-inference + Qdrant/LanceDB.☆65Updated 5 months ago
- A simple service that integrates vLLM with Ray Serve for fast and scalable LLM serving.☆67Updated last year
- Experimental Code for StructuredRAG: JSON Response Formatting with Large Language Models☆105Updated last month
- Code for evaluating with Flow-Judge-v0.1 - an open-source, lightweight (3.8B) language model optimized for LLM system evaluations. Crafte…☆70Updated 7 months ago
- Own your AI, search the web with it🌐😎☆85Updated 4 months ago
- Machine Learning Serving focused on GenAI with simplicity as the top priority.☆57Updated last month
- Simple examples using Argilla tools to build AI☆53Updated 6 months ago
- A Lightweight Library for AI Observability☆243Updated 3 months ago
- Solving data for LLMs - Create quality synthetic datasets!☆148Updated 4 months ago
- Python SDK for experimenting, testing, evaluating & monitoring LLM-powered applications - Parea AI (YC S23)☆77Updated 3 months ago
- Hugging Face Inference Toolkit used to serve transformers, sentence-transformers, and diffusers models.☆72Updated last week
- ☆156Updated this week
- Routing on Random Forest (RoRF)☆161Updated 8 months ago
- GenAIOps on Kubernetes: A collection of reference architectures for running GenAI at scale on Kubernetes using OSS tooling☆130Updated 7 months ago
- ☆149Updated 2 weeks ago
- ☆60Updated last year
- Dynamic Metadata based RAG Framework☆75Updated 10 months ago
- A stable, fast and easy-to-use inference library with a focus on a sync-to-async API☆45Updated 8 months ago
- Embed anything.☆28Updated last year
- ☆66Updated last year
- Dataset Viber is your chill repo for data collection, annotation and vibe checks.☆47Updated 8 months ago
- 🕹️ Performance Comparison of MLOps Engines, Frameworks, and Languages on Mainstream AI Models.☆137Updated 10 months ago
- This project enhances the construction of RAG applications by addressing challenges, improving accessibility, scalability, and managing d…☆145Updated last year
- Train Large Language Models on MLX.☆77Updated this week