bentoml / BentoVLLMLinks
Self-host LLMs with vLLM and BentoML
☆152Updated 2 weeks ago
Alternatives and similar repositories for BentoVLLM
Users that are interested in BentoVLLM are comparing it to the libraries listed below
Sorting:
- ☆64Updated 6 months ago
- Experimental Code for StructuredRAG: JSON Response Formatting with Large Language Models☆110Updated 6 months ago
- Tutorial for building LLM router☆231Updated last year
- Machine Learning Serving focused on GenAI with simplicity as the top priority.☆58Updated 2 weeks ago
- 🕹️ Performance Comparison of MLOps Engines, Frameworks, and Languages on Mainstream AI Models.☆139Updated last year
- Fine-tune an LLM to perform batch inference and online serving.☆112Updated 4 months ago
- Code for evaluating with Flow-Judge-v0.1 - an open-source, lightweight (3.8B) language model optimized for LLM system evaluations. Crafte…☆78Updated 11 months ago
- A collection of all available inference solutions for the LLMs☆91Updated 7 months ago
- A Lightweight Library for AI Observability☆251Updated 8 months ago
- GenAIOps on Kubernetes: A collection of reference architectures for running GenAI at scale on Kubernetes using OSS tooling☆134Updated 11 months ago
- Simple UI for debugging correlations of text embeddings☆296Updated 4 months ago
- ☆102Updated last year
- Benchmarking the serving capabilities of vLLM☆54Updated last year
- Route LLM requests to the best model for the task at hand.☆113Updated last month
- Routing on Random Forest (RoRF)☆214Updated last year
- Hugging Face Inference Toolkit used to serve transformers, sentence-transformers, and diffusers models.☆87Updated last month
- Own your AI, search the web with it🌐😎☆90Updated 9 months ago
- Accelerating your LLM training to full speed! Made with ❤️ by ServiceNow Research☆254Updated this week
- ☆169Updated last week
- This is the documentation repository for SGLang. It is auto-generated from https://github.com/sgl-project/sglang/tree/main/docs.☆82Updated this week
- Benchmark and optimize LLM inference across frameworks with ease☆124Updated last month
- Deployment a light and full OpenAI API for production with vLLM to support /v1/embeddings with all embeddings models.☆44Updated last year
- DSPY on action with OpenSource LLMs.☆97Updated last year
- Simple examples using Argilla tools to build AI☆56Updated 11 months ago
- Elasticsearch integration into LangChain☆63Updated 2 weeks ago
- A simple service that integrates vLLM with Ray Serve for fast and scalable LLM serving.☆73Updated last year
- A framework for fine-tuning retrieval-augmented generation (RAG) systems.☆131Updated this week
- vLLM adapter for a TGIS-compatible gRPC server.☆41Updated this week
- High level library for batched embeddings generation, blazingly-fast web-based RAG and quantized indexes processing ⚡☆67Updated 11 months ago
- ☆18Updated last year