runpod-workers / worker-vllmLinks
The RunPod worker template for serving our large language model endpoints. Powered by vLLM.
☆357Updated this week
Alternatives and similar repositories for worker-vllm
Users that are interested in worker-vllm are comparing it to the libraries listed below
Sorting:
- A fast batching API to serve LLM models☆185Updated last year
- function calling-based LLM agents☆288Updated 11 months ago
- The llama-cpp-agent framework is a tool designed for easy interaction with Large Language Models (LLMs). Allowing users to chat with LLM …☆586Updated 6 months ago
- 🐍 | Python library for RunPod API and serverless worker SDK.☆246Updated this week
- Examples of models deployable with Truss☆196Updated last week
- Generate Synthetic Data Using OpenAI, MistralAI or AnthropicAI☆222Updated last year
- This is our own implementation of 'Layer Selective Rank Reduction'☆240Updated last year
- One click templates for inferencing Language Models☆211Updated 3 weeks ago
- A simple Python sandbox for helpful LLM data agents☆279Updated last year
- Convenience scripts to finetune (chat-)LLaMa3 and other models for any language☆312Updated last year
- ☆161Updated 2 weeks ago
- A multimodal, function calling powered LLM webui.☆216Updated 11 months ago
- Low-Rank adapter extraction for fine-tuned transformers models☆175Updated last year
- TheBloke's Dockerfiles☆307Updated last year
- Tutorial for building LLM router☆224Updated last year
- ☆206Updated last year
- A benchmark for emotional intelligence in large language models☆347Updated last year
- Dataset Crafting w/ RAG/Wikipedia ground truth and Efficient Fine-Tuning Using MLX and Unsloth. Includes configurable dataset annotation …☆184Updated last year
- The easiest, and fastest way to run AI-generated Python code safely☆329Updated 8 months ago
- An OpenAI API compatible API for chat with image input and questions about the images. aka Multimodal.☆259Updated 5 months ago
- Fully-featured, beautiful web interface for vLLM - built with NextJS.☆150Updated 3 months ago
- Merge Transformers language models by use of gradient parameters.☆207Updated last year
- Falcon LLM ggml framework with CPU and GPU support☆247Updated last year
- Comparison of the output quality of quantization methods, using Llama 3, transformers, GGUF, EXL2.☆162Updated last year
- Dagger functions to import Hugging Face GGUF models into a local ollama instance and optionally push them to ollama.com.☆116Updated last year
- A tool for generating function arguments and choosing what function to call with local LLMs☆427Updated last year
- ☆199Updated last year
- ☆132Updated 4 months ago
- Convenient wrapper for fine-tuning and inference of Large Language Models (LLMs) with several quantization techniques (GTPQ, bitsandbytes…☆145Updated last year
- ☆1,060Updated 11 months ago