runpod-workers / worker-vllmLinks
The RunPod worker template for serving our large language model endpoints. Powered by vLLM.
☆371Updated 2 weeks ago
Alternatives and similar repositories for worker-vllm
Users that are interested in worker-vllm are comparing it to the libraries listed below
Sorting:
- A fast batching API to serve LLM models☆187Updated last year
- Examples of models deployable with Truss☆205Updated this week
- 🐍 | Python library for RunPod API and serverless worker SDK.☆253Updated this week
- function calling-based LLM agents☆287Updated last year
- Generate Synthetic Data Using OpenAI, MistralAI or AnthropicAI☆221Updated last year
- The llama-cpp-agent framework is a tool designed for easy interaction with Large Language Models (LLMs). Allowing users to chat with LLM …☆592Updated 7 months ago
- This is our own implementation of 'Layer Selective Rank Reduction'☆239Updated last year
- Convenience scripts to finetune (chat-)LLaMa3 and other models for any language☆314Updated last year
- A simple Python sandbox for helpful LLM data agents☆285Updated last year
- ☆207Updated last year
- TheBloke's Dockerfiles☆307Updated last year
- ☆162Updated 2 months ago
- ☆162Updated 7 months ago
- Tutorial for building LLM router☆228Updated last year
- Dagger functions to import Hugging Face GGUF models into a local ollama instance and optionally push them to ollama.com.☆117Updated last year
- Low-Rank adapter extraction for fine-tuned transformers models☆178Updated last year
- Dataset Crafting w/ RAG/Wikipedia ground truth and Efficient Fine-Tuning Using MLX and Unsloth. Includes configurable dataset annotation …☆185Updated last year
- ☆199Updated last year
- A multimodal, function calling powered LLM webui.☆216Updated last year
- One click templates for inferencing Language Models☆213Updated 2 months ago
- The easiest, and fastest way to run AI-generated Python code safely☆333Updated 10 months ago
- 🕹️ Performance Comparison of MLOps Engines, Frameworks, and Languages on Mainstream AI Models.☆138Updated last year
- ☆51Updated last year
- Convenient wrapper for fine-tuning and inference of Large Language Models (LLMs) with several quantization techniques (GTPQ, bitsandbytes…☆146Updated last year
- Merge Transformers language models by use of gradient parameters.☆208Updated last year
- Large-scale LLM inference engine☆1,558Updated this week
- Some simple scripts that I use day-to-day when working with LLMs and Huggingface Hub☆160Updated 2 years ago
- Fully-featured, beautiful web interface for vLLM - built with NextJS.☆157Updated 4 months ago
- Local LLM ReAct Agent with Guidance☆158Updated 2 years ago
- Function Calling Benchmark & Testing☆90Updated last year