runpod-workers / worker-vllm
The RunPod worker template for serving our large language model endpoints. Powered by vLLM.
☆299Updated this week
Alternatives and similar repositories for worker-vllm:
Users that are interested in worker-vllm are comparing it to the libraries listed below
- A fast batching API to serve LLM models☆183Updated 11 months ago
- The llama-cpp-agent framework is a tool designed for easy interaction with Large Language Models (LLMs). Allowing users to chat with LLM …☆547Updated last month
- 🐍 | Python library for RunPod API and serverless worker SDK.☆217Updated last week
- This is our own implementation of 'Layer Selective Rank Reduction'☆233Updated 10 months ago
- TheBloke's Dockerfiles☆305Updated last year
- A multimodal, function calling powered LLM webui.☆214Updated 6 months ago
- Low-Rank adapter extraction for fine-tuned transformers models☆171Updated 11 months ago
- ☆199Updated last year
- Comparison of the output quality of quantization methods, using Llama 3, transformers, GGUF, EXL2.☆148Updated 10 months ago
- ☆152Updated 8 months ago
- Function Calling Benchmark & Testing☆85Updated 8 months ago
- Domain Adapted Language Modeling Toolkit - E2E RAG☆318Updated 4 months ago
- Generate Synthetic Data Using OpenAI, MistralAI or AnthropicAI☆224Updated 11 months ago
- Evaluate and Enhance Your LLM Deployments for Real-World Inference Needs☆236Updated this week
- Local LLM ReAct Agent with Guidance☆158Updated last year
- ☆846Updated 6 months ago
- Convenient wrapper for fine-tuning and inference of Large Language Models (LLMs) with several quantization techniques (GTPQ, bitsandbytes…☆147Updated last year
- Convenience scripts to finetune (chat-)LLaMa3 and other models for any language☆300Updated 9 months ago
- function calling-based LLM agents☆285Updated 6 months ago
- A bagel, with everything.☆317Updated 11 months ago
- A benchmark for emotional intelligence in large language models☆253Updated 8 months ago
- Examples of models deployable with Truss☆167Updated this week
- Landmark Attention: Random-Access Infinite Context Length for Transformers QLoRA☆123Updated last year
- Dagger functions to import Hugging Face GGUF models into a local ollama instance and optionally push them to ollama.com.☆115Updated 10 months ago
- A simple Python sandbox for helpful LLM data agents☆241Updated 9 months ago
- Comparison of Language Model Inference Engines☆210Updated 3 months ago
- Customizable implementation of the self-instruct paper.☆1,040Updated last year
- Tutorial for building LLM router☆189Updated 8 months ago
- ☆125Updated last week
- ☆52Updated last year