SystemPanic / vllm-windowsLinks
A high-throughput and memory-efficient inference and serving engine for LLMs (Windows build & kernels)
☆100Updated last month
Alternatives and similar repositories for vllm-windows
Users that are interested in vllm-windows are comparing it to the libraries listed below
Sorting:
- ☆51Updated 8 months ago
- A pipeline parallel training script for LLMs.☆153Updated 2 months ago
- Game Companion AI is an advanced application designed to enhance the gaming experience by providing real-time analysis and interpretation…☆51Updated 9 months ago
- A Windows tool to query various LLM AIs. Supports branched conversations, history and summaries among others.☆33Updated last week
- Service for testing out the new Qwen2.5 omni model☆54Updated 2 months ago
- Wan2.1, quantized and optimized so it fits on your 3090/4090☆33Updated 4 months ago
- ☆39Updated 5 months ago
- Cosmos1GP for the GPU Poor by DeepBeepMeep☆71Updated 5 months ago
- automatically quant GGUF models☆187Updated this week
- Memory Management for the GPU Poor, run the latest open source frontier models on consumer Nvidia GPUs☆129Updated 2 weeks ago
- ACE-Step: A Step Towards Music Generation Foundation Model☆41Updated 2 months ago
- SoTA open-source TTS☆44Updated last month
- ☆116Updated 4 months ago
- The hearth of The Pulsar App, fast, secure and shared inference with modern UI☆57Updated 7 months ago
- 8-bit CUDA functions for PyTorch☆25Updated last year
- OminiControl for the GPU Poor☆35Updated 5 months ago
- Deploy Apollo HF space locally☆40Updated 7 months ago
- ☆116Updated 8 months ago
- win32 native frontend for llama-cli☆12Updated 8 months ago
- ☆203Updated 2 months ago
- A TTS model capable of generating ultra-realistic dialogue in one pass.☆31Updated 2 months ago
- A Lightweight Gradio Web interface for Text-to-Audio Generation utilising SAO1.0☆53Updated last year
- Development repository for the Triton language and compiler☆34Updated 9 months ago
- Run Orpheus 3B Locally with Gradio UI, Standalone App☆23Updated 3 months ago
- (Windows/Linux/MacOS) Local WebUI with neural network models (Text, Image, Video, 3D, Audio) on python (Gradio interface). Translated on …☆100Updated 2 weeks ago
- A simple wrapper around "F5-TTS: A Fairytaler that Fakes Fluent and Faithful Speech with Flow Matching" that provides an OpenAI-compatibl…☆14Updated 5 months ago
- This extension enhances the capabilities of textgen-webui by integrating advanced vision models, allowing users to have contextualized co…☆57Updated 9 months ago
- "a towel is about the most massively useful thing an interstellar AI hitchhiker can have"☆48Updated 9 months ago
- An AI focused photo manipulation tool based on Gradio☆185Updated 3 weeks ago
- Adding a multi-text multi-speaker script (diffe) that is based on a script from asiff00 on issue 61 for Sesame: A Conversational Speech G…☆23Updated 3 months ago