A high-throughput and memory-efficient inference and serving engine for LLMs
☆39Jun 24, 2026Updated last month
Alternatives and similar repositories for vllm
Users that are interested in vllm are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- ☆74Updated this week
- Official code for Guiding Language Model Math Reasoning with Planning Tokens☆19Feb 29, 2024Updated 2 years ago
- DeepSeek-V4 Lecture☆25Jun 5, 2026Updated last month
- [NeurIPS 25] InfiniPot-V: Memory-Constrained KV Cache Compression for Streaming Video Understanding☆20Jan 25, 2026Updated 5 months ago
- 2SSP: A Two-Stage Framework for Structured Pruning of LLMs☆21Aug 18, 2025Updated 11 months ago
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- EMNLP 2022: Analyzing and Evaluating Faithfulness in Dialogue Summarization☆13Mar 20, 2025Updated last year
- win32json Parser for TypeLibrary creation☆12Feb 10, 2022Updated 4 years ago
- An interactive companion toy that engages kids with storytelling, singing, and encouragement for physical activities using advanced AI t…☆10Oct 15, 2024Updated last year
- Code for paper: [ICLR2025 Oral] FlexPrefill: A Context-Aware Sparse Attention Mechanism for Efficient Long-Sequence Inference☆170Oct 13, 2025Updated 9 months ago
- Code for "Skill-based Chain-of-Thoughts for Domain-Adaptive Video Reasoning [EMNLP 2025 Findings]"☆18Aug 27, 2025Updated 10 months ago
- React app for inspecting, building and debugging with the Realtime API☆11Nov 5, 2024Updated last year
- Official implementation of A cappella: Audio-visual Singing VoiceSeparation, from BMVC21☆18May 14, 2022Updated 4 years ago
- ☆32May 26, 2024Updated 2 years ago
- deduplication☆15Feb 20, 2023Updated 3 years ago
- Bare Metal GPUs on DigitalOcean Gradient AI • AdPurpose-built for serious AI teams training foundational models, running large-scale inference, and pushing the boundaries of what's possible.
- Policy-driven seamless lazy loading☆29Updated this week
- 修复停止维护的Vxhunter IDA插件项目在python3高些的版本下运行的若干问题☆14Aug 26, 2024Updated last year
- Dev Ubuntu 20.04 Docker image, with Oh-My-Zsh/Pyenv/GNU&LLVM Build Tools, suitable as a temp sysroot☆12Aug 10, 2022Updated 3 years ago
- Resource scheduling and cluster management for AI☆15Jun 29, 2023Updated 3 years ago
- Binary Ninja plugin for forward/backward slicing☆18Nov 25, 2020Updated 5 years ago
- Code for EMNLP 2022 main conference paper "Information-Transport-based Policy for Simultaneous Translation"☆13Nov 3, 2022Updated 3 years ago
- ☆76Mar 26, 2025Updated last year
- Waf based on caddy2☆21Jul 21, 2022Updated 4 years ago
- 基于Qt5的网络音乐播放器，调的酷狗的API（但是现在好像已经不能用了）。歌词显示、搜索歌曲、桌面歌词等☆12Dec 18, 2019Updated 6 years ago
- Managed Kubernetes at scale on DigitalOcean • AdDigitalOcean Kubernetes includes the control plane, bandwidth allowance, container registry, automatic updates, and more for free.
- [TAI 2025] Official implementation of TAI-accepted paper: ShadowMaskFormer: Mask Augmented Patch Embedding for Shadow Removal☆15May 8, 2025Updated last year
- Julia implementation of flash-attention operation for neural networks.☆11May 31, 2023Updated 3 years ago
- Sparse Attention with Linear Units☆20Apr 21, 2021Updated 5 years ago
- Community maintained hardware plugin for vLLM on Ascend☆2,461Updated this week
- A simple C Thread pool implementation☆13Apr 10, 2020Updated 6 years ago
- Agent skills for VulHunt☆15Mar 7, 2026Updated 4 months ago
- ☆79Dec 15, 2023Updated 2 years ago
- Open-source solution for reliable, cloud-native MariaDB database backups to S3-compatible storage. 🛡️ Easy setup with environment variab…☆15Jul 5, 2026Updated 2 weeks ago
- Sparse symmetric indefinite solver implemented with a runtime system☆13May 11, 2020Updated 6 years ago
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- Automatic differentiation of FEniCS and Firedrake models in Julia☆14Mar 21, 2021Updated 5 years ago
- SQL Optimizations using MLIR☆12Apr 5, 2020Updated 6 years ago
- a ros node using face_net do face_recognition☆12Jul 27, 2016Updated 9 years ago
- ocr照片识别文字,包括裁剪图片,能识别中文和英文,是现有网上资源中识别率最好的☆14Sep 20, 2016Updated 9 years ago
- ☆47Apr 10, 2026Updated 3 months ago
- NJUNMT for docNMT☆16Sep 9, 2020Updated 5 years ago
- Code to obtain the training data for the ACL 2018 paper "Neural Document Summarization by Jointly Learning to Score and Select Sentences…☆17Jul 5, 2019Updated 7 years ago