runpod-workers / worker-vllm
The RunPod worker template for serving our large language model endpoints. Powered by vLLM.
☆244Updated 2 weeks ago
Related projects ⓘ
Alternatives and complementary repositories for worker-vllm
- A fast batching API to serve LLM models☆172Updated 6 months ago
- TheBloke's Dockerfiles☆299Updated 8 months ago
- Generate Synthetic Data Using OpenAI, MistralAI or AnthropicAI☆221Updated 6 months ago
- ☆52Updated last year
- This is our own implementation of 'Layer Selective Rank Reduction'☆232Updated 5 months ago
- Low-Rank adapter extraction for fine-tuned transformers model☆162Updated 6 months ago
- Harnessing the Memory Power of the Camelids☆145Updated last year
- Landmark Attention: Random-Access Infinite Context Length for Transformers QLoRA☆124Updated last year
- A multimodal, function calling powered LLM webui.☆208Updated last month
- 🐍 | Python library for RunPod API and serverless worker SDK.☆182Updated this week
- function calling-based LLM agents☆278Updated 2 months ago
- Merge Transformers language models by use of gradient parameters.☆201Updated 3 months ago
- Convenient wrapper for fine-tuning and inference of Large Language Models (LLMs) with several quantization techniques (GTPQ, bitsandbytes…☆145Updated last year
- ☆112Updated this week
- Local LLM ReAct Agent with Guidance☆155Updated last year
- Dataset Crafting w/ RAG/Wikipedia ground truth and Efficient Fine-Tuning Using MLX and Unsloth. Includes configurable dataset annotation …☆162Updated 4 months ago
- QLoRA: Efficient Finetuning of Quantized LLMs☆77Updated 7 months ago
- Domain Adapted Language Modeling Toolkit - E2E RAG☆311Updated last week
- An unsupervised model merging algorithm for Transformers-based language models.☆100Updated 6 months ago
- Web UI for ExLlamaV2☆445Updated last month
- Dagger functions to import Hugging Face GGUF models into a local ollama instance and optionally push them to ollama.com.☆110Updated 5 months ago
- ☆168Updated last year
- idea: https://github.com/nyxkrage/ebook-groupchat/☆82Updated 3 months ago
- Comparison of the output quality of quantization methods, using Llama 3, transformers, GGUF, EXL2.☆126Updated 6 months ago
- Some simple scripts that I use day-to-day when working with LLMs and Huggingface Hub☆155Updated last year
- Memoir+ a persona extension for Text Gen Web UI. That includes memory, emotions, command handling and more.☆171Updated last month
- ☆104Updated 8 months ago
- Falcon LLM ggml framework with CPU and GPU support☆244Updated 9 months ago
- ☆149Updated 4 months ago
- Auto Data is a library designed for quick and effortless creation of datasets tailored for fine-tuning Large Language Models (LLMs).☆89Updated 2 weeks ago