bentoml / BentoVLLMLinks
Self-host LLMs with vLLM and BentoML
☆150Updated this week
Alternatives and similar repositories for BentoVLLM
Users that are interested in BentoVLLM are comparing it to the libraries listed below
Sorting:
- Tutorial for building LLM router☆228Updated last year
- ☆64Updated 6 months ago
- Experimental Code for StructuredRAG: JSON Response Formatting with Large Language Models☆111Updated 5 months ago
- 🕹️ Performance Comparison of MLOps Engines, Frameworks, and Languages on Mainstream AI Models.☆138Updated last year
- Machine Learning Serving focused on GenAI with simplicity as the top priority.☆59Updated 2 months ago
- A Lightweight Library for AI Observability☆251Updated 7 months ago
- ☆102Updated last year
- Fine-tune an LLM to perform batch inference and online serving.☆112Updated 4 months ago
- Simple UI for debugging correlations of text embeddings☆292Updated 4 months ago
- Routing on Random Forest (RoRF)☆211Updated last year
- A collection of all available inference solutions for the LLMs☆91Updated 7 months ago
- Accelerating your LLM training to full speed! Made with ❤️ by ServiceNow Research☆237Updated this week
- GenAIOps on Kubernetes: A collection of reference architectures for running GenAI at scale on Kubernetes using OSS tooling☆134Updated 11 months ago
- Solving data for LLMs - Create quality synthetic datasets!☆151Updated 8 months ago
- A simple service that integrates vLLM with Ray Serve for fast and scalable LLM serving.☆72Updated last year
- ☆66Updated last year
- Deployment a light and full OpenAI API for production with vLLM to support /v1/embeddings with all embeddings models.☆43Updated last year
- Code for evaluating with Flow-Judge-v0.1 - an open-source, lightweight (3.8B) language model optimized for LLM system evaluations. Crafte…☆78Updated 11 months ago
- TitanML Takeoff Server is an optimization, compression and deployment platform that makes state of the art machine learning models access…☆114Updated last year
- Own your AI, search the web with it🌐😎☆90Updated 8 months ago
- ☆232Updated 3 months ago
- This is the documentation repository for SGLang. It is auto-generated from https://github.com/sgl-project/sglang/tree/main/docs.☆78Updated this week
- ☆77Updated 8 months ago
- High level library for batched embeddings generation, blazingly-fast web-based RAG and quantized indexes processing ⚡☆67Updated 11 months ago
- Unattended Lightweight Text Classifiers with LLM Embeddings☆184Updated last year
- [ACL'25] Official Code for LlamaDuo: LLMOps Pipeline for Seamless Migration from Service LLMs to Small-Scale Local LLMs☆314Updated 2 months ago
- Using LlamaIndex with Ray for productionizing LLM applications☆71Updated 2 years ago
- ☆210Updated 3 months ago
- Hugging Face Inference Toolkit used to serve transformers, sentence-transformers, and diffusers models.☆87Updated 2 weeks ago
- DSPY on action with OpenSource LLMs.☆94Updated last year