SystemPanic / vllm-windowsLinks
A high-throughput and memory-efficient inference and serving engine for LLMs (Windows build & kernels)
☆156Updated 2 weeks ago
Alternatives and similar repositories for vllm-windows
Users that are interested in vllm-windows are comparing it to the libraries listed below
Sorting:
- ☆121Updated 10 months ago
- A simple wrapper around "F5-TTS: A Fairytaler that Fakes Fluent and Faithful Speech with Flow Matching" that provides an OpenAI-compatibl…☆15Updated 7 months ago
- Service for testing out the new Qwen2.5 omni model☆57Updated 4 months ago
- Quantized text-audio foundation model from Boson AI☆33Updated last month
- Deepspeed windows information☆42Updated last year
- A Windows tool to query various LLM AIs. Supports branched conversations, history and summaries among others.☆33Updated this week
- automatically quant GGUF models☆199Updated last week
- ☆51Updated 10 months ago
- ☆41Updated 7 months ago
- The hearth of The Pulsar App, fast, secure and shared inference with modern UI☆57Updated 9 months ago
- This is a pre-built wheel of Triton 3.3.0 for Windows with Nvidia only + Proton☆30Updated 3 months ago
- SoTA open-source TTS☆84Updated last week
- Game Companion AI is an advanced application designed to enhance the gaming experience by providing real-time analysis and interpretation…☆53Updated 11 months ago
- ☆40Updated 7 months ago
- Polyglot is a fast, elegant, and free translation tool using AI.☆62Updated last year
- Run Orpheus 3B Locally with Gradio UI, Standalone App☆22Updated 5 months ago
- ☆124Updated 6 months ago
- A pipeline parallel training script for LLMs.☆158Updated 4 months ago
- Fast and memory-efficient exact attention - Windows wheels☆36Updated 4 months ago
- Simple UI for Llama-3.2-11B-Vision & Molmo-7B-D☆137Updated 11 months ago
- Croco.Cpp is fork of KoboldCPP infering GGML/GGUF models on CPU/Cuda with KoboldAI's UI. It's powered partly by IK_LLama.cpp, and compati…☆137Updated this week
- This extension enhances the capabilities of textgen-webui by integrating advanced vision models, allowing users to have contextualized co…☆57Updated 10 months ago
- Writing Extension for Text Generation WebUI☆63Updated last month
- ACE-Step: A Step Towards Music Generation Foundation Model☆43Updated 3 months ago
- Free ComfyUI Workflows☆34Updated last week
- (Windows/Linux/MacOS) Local WebUI with neural network models (Text, Image, Video, 3D, Audio) on python (Gradio interface). Translated on …☆101Updated 2 weeks ago
- ☆211Updated 4 months ago
- stable-diffusion.cpp bindings for python☆62Updated last week
- ☆50Updated 6 months ago
- Memory Management for the GPU Poor, run the latest open source frontier models on consumer Nvidia GPUs☆148Updated last week