substratusai / vllm-docker
☆53Updated last month
Alternatives and similar repositories for vllm-docker:
Users that are interested in vllm-docker are comparing it to the libraries listed below
- Self-host LLMs with vLLM and BentoML☆87Updated this week
- ☆18Updated 6 months ago
- experiments with inference on llama☆104Updated 8 months ago
- An OpenAI Completions API compatible server for NLP transformers models☆64Updated last year
- Machine Learning Serving focused on GenAI with simplicity as the top priority.☆58Updated last month
- The backend behind the LLM-Perf Leaderboard☆10Updated 9 months ago
- A simple service that integrates vLLM with Ray Serve for fast and scalable LLM serving.☆62Updated 10 months ago
- 🕹️ Performance Comparison of MLOps Engines, Frameworks, and Languages on Mainstream AI Models.☆136Updated 6 months ago
- Benchmark suite for LLMs from Fireworks.ai☆66Updated last week
- ☆53Updated 8 months ago
- Ready-to-go containerized RAG service. Implemented with text-embedding-inference + Qdrant/LanceDB.☆57Updated last month
- Experimental Code for StructuredRAG: JSON Response Formatting with Large Language Models☆100Updated 2 months ago
- Evaluate and Enhance Your LLM Deployments for Real-World Inference Needs☆191Updated this week
- High level library for batched embeddings generation, blazingly-fast web-based RAG and quantized indexes processing ⚡☆64Updated 3 months ago
- ☆20Updated last year
- Code for NeurIPS LLM Efficiency Challenge☆55Updated 10 months ago
- A framework for evaluating function calls made by LLMs☆36Updated 6 months ago
- Using LlamaIndex with Ray for productionizing LLM applications☆71Updated last year
- 🚀 Scale your RAG pipeline using Ragswift: A scalable centralized embeddings management platform☆37Updated last year
- Client Code Examples, Use Cases and Benchmarks for Enterprise h2oGPTe RAG-Based GenAI Platform☆82Updated last week
- Data preparation code for Amber 7B LLM☆85Updated 9 months ago
- IBM development fork of https://github.com/huggingface/text-generation-inference☆59Updated 2 months ago
- ☆199Updated last year
- LLM reads a paper and produce a working prototype☆48Updated 2 weeks ago
- ☆30Updated 7 months ago
- TitanML Takeoff Server is an optimization, compression and deployment platform that makes state of the art machine learning models access…☆114Updated last year
- ☆159Updated this week
- Deployment a light and full OpenAI API for production with vLLM to support /v1/embeddings with all embeddings models.☆40Updated 7 months ago
- GPT-4 Level Conversational QA Trained In a Few Hours☆58Updated 6 months ago
- C++ inference wrappers for running blazing fast embedding services on your favourite serverless like AWS Lambda. By Prithivi Da, PRs welc…☆21Updated 11 months ago