runpod-workers / worker-vllmLinks
The RunPod worker template for serving our large language model endpoints. Powered by vLLM.
☆342Updated this week
Alternatives and similar repositories for worker-vllm
Users that are interested in worker-vllm are comparing it to the libraries listed below
Sorting:
- A fast batching API to serve LLM models☆185Updated last year
- Examples of models deployable with Truss☆192Updated this week
- 🐍 | Python library for RunPod API and serverless worker SDK.☆244Updated this week
- function calling-based LLM agents☆287Updated 10 months ago
- The llama-cpp-agent framework is a tool designed for easy interaction with Large Language Models (LLMs). Allowing users to chat with LLM …☆579Updated 5 months ago
- ☆205Updated last year
- TheBloke's Dockerfiles☆305Updated last year
- This is our own implementation of 'Layer Selective Rank Reduction'☆239Updated last year
- A simple Python sandbox for helpful LLM data agents☆277Updated last year
- Convenience scripts to finetune (chat-)LLaMa3 and other models for any language☆310Updated last year
- Generate Synthetic Data Using OpenAI, MistralAI or AnthropicAI☆222Updated last year
- Tutorial for building LLM router☆221Updated last year
- One click templates for inferencing Language Models☆201Updated 2 weeks ago
- Dataset Crafting w/ RAG/Wikipedia ground truth and Efficient Fine-Tuning Using MLX and Unsloth. Includes configurable dataset annotation …☆184Updated last year
- A multimodal, function calling powered LLM webui.☆215Updated 10 months ago
- A benchmark for emotional intelligence in large language models☆336Updated last year
- Low-Rank adapter extraction for fine-tuned transformers models☆175Updated last year
- A bagel, with everything.☆323Updated last year
- ☆199Updated last year
- Merge Transformers language models by use of gradient parameters.☆206Updated 11 months ago
- Convenient wrapper for fine-tuning and inference of Large Language Models (LLMs) with several quantization techniques (GTPQ, bitsandbytes…☆146Updated last year
- A Lightweight Library for AI Observability☆250Updated 5 months ago
- An OpenAI-like LLaMA inference API☆112Updated last year
- Domain Adapted Language Modeling Toolkit - E2E RAG☆325Updated 8 months ago
- Dagger functions to import Hugging Face GGUF models into a local ollama instance and optionally push them to ollama.com.☆116Updated last year
- FineTune LLMs in few lines of code (Text2Text, Text2Speech, Speech2Text)☆240Updated last year
- ☆157Updated last year
- The easiest, and fastest way to run AI-generated Python code safely☆330Updated 8 months ago
- ☆52Updated last year
- 🕹️ Performance Comparison of MLOps Engines, Frameworks, and Languages on Mainstream AI Models.☆137Updated last year