runpod-workers / worker-vllmLinks
The RunPod worker template for serving our large language model endpoints. Powered by vLLM.
☆373Updated last week
Alternatives and similar repositories for worker-vllm
Users that are interested in worker-vllm are comparing it to the libraries listed below
Sorting:
- A fast batching API to serve LLM models☆188Updated last year
- function calling-based LLM agents☆289Updated last year
- Examples of models deployable with Truss☆206Updated this week
- The llama-cpp-agent framework is a tool designed for easy interaction with Large Language Models (LLMs). Allowing users to chat with LLM …☆601Updated 8 months ago
- TheBloke's Dockerfiles☆306Updated last year
- 🐍 | Python library for RunPod API and serverless worker SDK.☆254Updated 2 weeks ago
- Convenience scripts to finetune (chat-)LLaMa3 and other models for any language☆316Updated last year