vllm-project / vllm-omniLinks
A framework for efficient model inference with omni-modality models
☆2,491Updated this week
Alternatives and similar repositories for vllm-omni
Users that are interested in vllm-omni are comparing it to the libraries listed below
Sorting:
- A compact implementation of SGLang, designed to demystify the complexities of modern LLM serving systems.☆3,297Updated 2 weeks ago
- Checkpoint-engine is a simple middleware to update model weights in LLM inference engines☆902Updated this week
- Conditional Memory via Scalable Lookup: A New Axis of Sparsity for Large Language Models☆3,545Updated 3 weeks ago
- VeOmni: Scaling Any Modality Model Training with Model-Centric Distributed Recipe Zoo☆1,601Updated this week
- ☆1,278Updated 2 months ago
- Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM☆2,660Updated this week
- Supercharge Your LLM with the Fastest KV Cache Layer☆6,787Updated last week
- A fast communication-overlapping library for tensor/expert parallelism on GPUs.☆1,235Updated 5 months ago
- Scalable toolkit for efficient model reinforcement☆1,293Updated this week
- Muon is Scalable for LLM Training☆1,421Updated 6 months ago
- WeDLM: The fastest diffusion language model with standard causal attention and native KV cache compatibility, delivering real speedups ov…☆597Updated 3 weeks ago
- slime is an LLM post-training framework for RL Scaling.☆3,571Updated last week
- MoBA: Mixture of Block Attention for Long-Context LLMs☆2,038Updated 10 months ago
- ☆1,282Updated 2 weeks ago
- ☆1,465Updated 2 months ago
- vLLM’s reference system for K8S-native cluster-wide deployment with community-driven performance optimization☆2,140Updated this week
- ☆725Updated 2 months ago
- Qwen3-omni is a natively end-to-end, omni-modal LLM developed by the Qwen team at Alibaba Cloud, capable of understanding text, audio, im…☆3,366Updated 3 weeks ago
- 🤗 A PyTorch-native and Flexible Inference Engine with Hybrid Cache Acceleration and Parallelism for DiTs.☆945Updated this week
- Fast, Flexible and Portable Structured Generation☆1,511Updated 2 weeks ago
- GLM-4.6V/4.5V/4.1V-Thinking: Towards Versatile Multimodal Reasoning with Scalable Reinforcement Learning☆2,145Updated last week
- FlashInfer: Kernel Library for LLM Serving☆4,853Updated this week
- LLM model quantization (compression) toolkit with hw acceleration support for Nvidia CUDA, AMD ROCm, Intel XPU and Intel/AMD/Apple CPU vi…☆1,007Updated this week
- Train speculative decoding models effortlessly and port them smoothly to SGLang serving.☆676Updated this week
- Common recipes to run vLLM☆364Updated this week
- cuTile is a programming model for writing parallel kernels for NVIDIA GPUs☆1,878Updated last week
- Kimi-VL: Mixture-of-Experts Vision-Language Model for Multimodal Reasoning, Long-Context Understanding, and Strong Agent Capabilities☆1,151Updated 6 months ago
- Virtualized Elastic KV Cache for Dynamic GPU Sharing and Beyond☆765Updated 3 weeks ago
- DFloat11 [NeurIPS '25]: Lossless Compression of LLMs and DiTs for Efficient GPU Inference☆600Updated 2 months ago
- [NeurIPS'24 Spotlight, ICLR'25, ICML'25] To speed up Long-context LLMs' inference, approximate and dynamic sparse calculate the attention…☆1,180Updated 4 months ago