substratusai / vllm-docker
☆47Updated this week
Related projects ⓘ
Alternatives and complementary repositories for vllm-docker
- A simple service that integrates vLLM with Ray Serve for fast and scalable LLM serving.☆54Updated 7 months ago
- Self-host LLMs with vLLM and BentoML☆74Updated last week
- Evaluate and Enhance Your LLM Deployments for Real-World Inference Needs☆168Updated 2 weeks ago
- Deployment a light and full OpenAI API for production with vLLM to support /v1/embeddings with all embeddings models.☆37Updated 4 months ago
- Machine Learning Serving focused on GenAI with simplicity as the top priority.☆57Updated 4 months ago
- ☆53Updated 5 months ago
- Using LlamaIndex with Ray for productionizing LLM applications☆71Updated last year
- ☆37Updated 11 months ago
- A stable, fast and easy-to-use inference library with a focus on a sync-to-async API☆45Updated last month
- ☆200Updated 9 months ago
- vLLM: A high-throughput and memory-efficient inference and serving engine for LLMs☆89Updated this week
- ☆104Updated 8 months ago
- ☆94Updated 2 months ago
- TitanML Takeoff Server is an optimization, compression and deployment platform that makes state of the art machine learning models access…☆114Updated 10 months ago
- IBM development fork of https://github.com/huggingface/text-generation-inference☆57Updated last month
- Experimental Code for StructuredRAG: Structured Outputs in Retrieval-Augmented Generation☆94Updated this week
- 🕹️ Performance Comparison of MLOps Engines, Frameworks, and Languages on Mainstream AI Models.☆134Updated 3 months ago
- Code for evaluating with Flow-Judge-v0.1 - an open-source, lightweight (3.8B) language model optimized for LLM system evaluations. Crafte…☆53Updated 3 weeks ago
- The backend behind the LLM-Perf Leaderboard☆10Updated 6 months ago
- Simple examples using Argilla tools to build AI☆42Updated this week
- experiments with inference on llama☆105Updated 5 months ago
- ☆64Updated 5 months ago
- LLM reads a paper and produce a working prototype☆36Updated 2 weeks ago
- 🚀 Scale your RAG pipeline using Ragswift: A scalable centralized embeddings management platform☆36Updated 9 months ago
- Google TPU optimizations for transformers models☆75Updated this week
- GPT-4 Level Conversational QA Trained In a Few Hours☆55Updated 3 months ago
- A guidance compatibility layer for llama-cpp-python☆34Updated last year
- Routing on Random Forest (RoRF)☆84Updated 2 months ago
- ☆16Updated 3 months ago
- Python client library for improving your LLM app accuracy☆96Updated this week