SystemPanic / vllm-windowsLinks
A high-throughput and memory-efficient inference and serving engine for LLMs (Windows build & kernels)
☆195Updated last week
Alternatives and similar repositories for vllm-windows
Users that are interested in vllm-windows are comparing it to the libraries listed below
Sorting:
- ☆41Updated 8 months ago
- ☆126Updated 7 months ago
- Service for testing out the new Qwen2.5 omni model☆61Updated 6 months ago
- ☆124Updated 11 months ago
- Quantized text-audio foundation model from Boson AI☆38Updated 2 months ago
- This is a pre-built wheel of Triton 3.3.0 for Windows with Nvidia only + Proton☆38Updated 5 months ago
- SoTA open-source TTS☆112Updated last week
- ☆42Updated 8 months ago
- ☆51Updated 11 months ago
- llama.cpp fork with additional SOTA quants and improved performance☆34Updated this week
- Deepspeed windows information☆42Updated last year
- automatically quant GGUF models☆214Updated last week
- Docker compose to run vLLM on Windows☆103Updated last year
- Make abliterated models with transformers, easy and fast☆90Updated 6 months ago
- (Windows/Linux/MacOS) Local WebUI with neural network models (Text, Image, Video, 3D, Audio) on python (Gradio interface). Translated on …☆105Updated last week
- Free ComfyUI Workflows☆37Updated last month
- Lightweight Gradio based WebUI for orpheusTTS - WSL / Linux [CUDA]☆106Updated 7 months ago
- Croco.Cpp is fork of KoboldCPP infering GGML/GGUF models on CPU/Cuda with KoboldAI's UI. It's powered partly by IK_LLama.cpp, and compati…☆152Updated this week
- A collection of compiled wheels for deepspeed built for python 3.10 and 3.11 with support for cuda 11.8 and 12.1 for Windows☆73Updated last year
- Fast and memory-efficient exact attention☆17Updated 2 weeks ago
- OminiControl for the GPU Poor☆39Updated 9 months ago
- Run Orpheus 3B Locally with Gradio UI, Standalone App☆21Updated 6 months ago
- ☆218Updated 5 months ago
- Game Companion AI is an advanced application designed to enhance the gaming experience by providing real-time analysis and interpretation…☆53Updated last year
- Orpheus Chat WebUI☆74Updated 7 months ago
- Speech-to-speech AI assistant with natural conversation flow, mid-speech interruption, vision capabilities and AI-initiated follow-ups. F…☆252Updated 6 months ago
- ACE-Step: A Step Towards Music Generation Foundation Model☆45Updated 5 months ago
- A Windows tool to query various LLM AIs. Supports branched conversations, history and summaries among others.☆32Updated last week
- A simple wrapper around "F5-TTS: A Fairytaler that Fakes Fluent and Faithful Speech with Flow Matching" that provides an OpenAI-compatibl…☆15Updated 8 months ago
- Prompt-based Evolutionary Nudity Iteration System☆135Updated 3 months ago