sasha0552 / vllm-ci

Related projects ⓘ

Alternatives and complementary repositories for vllm-ci

• itsme2417 / PolyMind
  A multimodal, function calling powered LLM webui.
  ☆208Updated last month
• theroyallab / tabbyAPI
  An OAI compatible exllamav2 API that's both lightweight and fast
  ☆605Updated this week
• turboderp / exui
  Web UI for ExLlamaV2
  ☆445Updated last month
• chu-tianxiang / vllm-gptq
  A high-throughput and memory-efficient inference and serving engine for LLMs
  ☆130Updated 4 months ago
• brucepro / Memoir
  Memoir+ a persona extension for Text Gen Web UI. That includes memory, emotions, command handling and more.
  ☆171Updated last month
• epolewski / EricLLM
  A fast batching API to serve LLM models
  ☆172Updated 6 months ago
• PygmalionAI / aphrodite-engine
  Large-scale LLM inference engine
  ☆1,140Updated this week
• kvcache-ai / ktransformers
  A Flexible Framework for Experiencing Cutting-edge LLM Inference Optimizations
  ☆737Updated last week
• cduk / vllm-pascal
  A fork of vLLM enabling Pascal architecture GPUs
  ☆20Updated last week
• lapp0 / lm-inference-engines
  Comparison of Language Model Inference Engines
  ☆190Updated 2 months ago
• runpod-workers / worker-vllm
  The RunPod worker template for serving our large language model endpoints. Powered by vLLM.
  ☆245Updated this week
• tdrussell / qlora-pipe
  A pipeline parallel training script for LLMs.
  ☆83Updated this week
• matt-c1 / llama-3-quant-comparison
  Comparison of the output quality of quantization methods, using Llama 3, transformers, GGUF, EXL2.
  ☆126Updated 6 months ago
• arcee-ai / PruneMe
  Automated Identification of Redundant Layer Blocks for Pruning in Large Language Models
  ☆196Updated 6 months ago
• IST-DASLab / marlin
  FP16xINT4 LLM inference kernel that can achieve near-ideal ~4x speedups up to medium batchsizes of 16-32 tokens.
  ☆624Updated 2 months ago
• mamei16 / LLM_Web_search
  An extension for oobabooga/text-generation-webui that enables the LLM to search the web using DuckDuckGo
  ☆173Updated this week
• LostRuins / datasetexplorer
  Easily view and modify JSON datasets for large language models
  ☆62Updated last month
• cognitivecomputations / laserRMT
  This is our own implementation of 'Layer Selective Rank Reduction'
  ☆232Updated 5 months ago
• ModelCloud / GPTQModel
  Production ready LLM model compression/quantization toolkit with accelerated inference support for both cpu/gpu via HF, vLLM, and SGLang.
  ☆125Updated this week
• jondurbin / bagel
  A bagel, with everything.
  ☆312Updated 7 months ago
• npuichigo / openai_trtllm
  OpenAI compatible API for TensorRT LLM triton backend
  ☆177Updated 3 months ago
• limcheekin / open-text-embeddings
  Open Source Text Embedding Models with OpenAI Compatible API
  ☆131Updated 4 months ago
• lmg-anon / mikupad
  LLM Frontend in a single html file
  ☆259Updated 2 weeks ago
• Leeroo-AI / mergoo
  A library for easily merging multiple LLM experts, and efficiently train the merged LLM.
  ☆408Updated 2 months ago
• mobiusml / hqq
  Official implementation of Half-Quadratic Quantization (HQQ)
  ☆702Updated this week
• matatonic / openedai-vision
  An OpenAI API compatible API for chat with image input and questions about the images. aka Multimodal.
  ☆203Updated last month
• Maximilian-Winter / llama-cpp-agent
  The llama-cpp-agent framework is a tool designed for easy interaction with Large Language Models (LLMs). Allowing users to chat with LLM …
  ☆494Updated 3 months ago
• Gryphe / MergeMonster
  An unsupervised model merging algorithm for Transformers-based language models.
  ☆100Updated 6 months ago
• rombodawg / Easy_training
  ☆40Updated 2 months ago
• v2rockets / Loyal-Elephie
  Your Trusty Memory-enabled AI Companion - Simple RAG chatbot optimized for local LLMs | 12 Languages Supported | OpenAI API Compatible
  ☆263Updated 2 months ago