A high-throughput and memory-efficient inference and serving engine for LLMs
☆10Jul 22, 2023Updated 3 years ago
Alternatives and similar repositories for vllm
Users that are interested in vllm are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Transformer related optimization, including BERT, GPT☆17Jul 29, 2023Updated 3 years ago
- CuteDSL Swizzle Visualize Tool☆14Feb 22, 2026Updated 8 months ago
- A curated list for Efficient Large Language Models☆11Mar 25, 2024Updated 2 years ago
- Optimize QWen1.5 models with TensorRT-LLM☆17May 14, 2024Updated 2 years ago
- ☆15Feb 23, 2025Updated last year
- Managed Database hosting by DigitalOcean • AdPostgreSQL, MySQL, MongoDB, Kafka, Valkey, and OpenSearch available. Automatically scale up storage and focus on building your apps.
- ☆13Mar 28, 2025Updated last year
- 基于SpringCloud(Hoxton.SR7) + SpringBoot(2.2.9.RELEASE) 的SaaS型微服务快速开发平台，具有统一授权、认证后台管理系统，其中包含具备用户管理、资源权限管理、网关API支持白名单、分布式事务等多个模块，支持多业务系统并行开发…☆19Oct 8, 2020Updated 6 years ago
- ☆15Jun 26, 2024Updated 2 years ago
- ☆30Mar 27, 2023Updated 3 years ago
- 📚Some papers & books I’ve read.☆31May 7, 2026Updated 5 months ago
- ffmpeg+cuvid+tensorrt+multicamera☆12Dec 31, 2024Updated last year
- Category independent sequential image segmentation. Find a segment containing a given image pixel within a given attention mask using a …☆13Oct 11, 2021Updated 5 years ago
- Evaluation tool for object detection models☆15Jan 6, 2026Updated 9 months ago
- 基于Vue+Element-ui+axios开发的一套电商平台前端模板☆21Oct 24, 2020Updated 6 years ago
- Virtual machines for every use case on DigitalOcean • AdGet dependable uptime with 99.99% SLA, simple security tools, and predictable monthly pricing with DigitalOcean's virtual machines, called Droplets.
- ACEnet: Anatomical Context-Encoding Network for Neuroanatomy Segmentation☆16Feb 14, 2022Updated 4 years ago
- ☆19Mar 4, 2025Updated last year
- a 3D FPN backbone for object detection☆15Nov 7, 2018Updated 7 years ago
- 基于MNN和cpp-httplib实现的简单llm推理服务☆10Apr 9, 2024Updated 2 years ago
- A Typescript API client for AUTOMATIC111/stable-diffusion-webui☆11Oct 18, 2023Updated 3 years ago
- Simple and efficient memory pool is implemented with C++11.☆10Jun 2, 2022Updated 4 years ago
- 异步命令组件☆21Aug 1, 2023Updated 3 years ago
- Allows you to use a macro prefix_all to prefix every attribute in structs and enums on serialization☆12Mar 28, 2021Updated 5 years ago
- MYSQL 监控工具，优化工具，各种工具为一体的java spring boot 项目☆26Jun 21, 2022Updated 4 years ago
- Virtual machines for every use case on DigitalOcean • AdGet dependable uptime with 99.99% SLA, simple security tools, and predictable monthly pricing with DigitalOcean's virtual machines, called Droplets.
- An open implementation for Feishu Skills, allowing agents to send messages, write documents, managing TODOs for you.☆14Feb 28, 2026Updated 8 months ago
- The reproduction of SRCNN method for super-resolution☆31Sep 5, 2019Updated 7 years ago
- A high-throughput and memory-efficient inference and serving engine for LLMs☆17Jun 3, 2024Updated 2 years ago
- Summary of system papers/frameworks/codes/tools on training or serving large model☆57Dec 17, 2023Updated 2 years ago
- An auxiliary project analysis of the characteristics of KV in DiT Attention.☆34Nov 29, 2024Updated last year
- WebAssembly Micro Runtime (WAMR)☆11Aug 1, 2025Updated last year
- envd Website☆12Sep 29, 2026Updated last month
- code for MM-WHS 2017: Multi-Modality Whole Heart Segmentation☆15Sep 1, 2017Updated 9 years ago
- ☆62Nov 21, 2024Updated last year
- Managed Kubernetes at scale on DigitalOcean • AdDigitalOcean Kubernetes includes the control plane, bandwidth allowance, container registry, automatic updates, and more for free.
- Send and receive OSC data to and from bevy and other programs or controllers☆11Jul 28, 2024Updated 2 years ago
- Top-2 Solution for CVPR UG2+ Track2☆18May 25, 2022Updated 4 years ago
- ☆13Sep 22, 2025Updated last year
- Extract frontmatter from markdown code blocks using remark, and do interesting things!☆13Dec 4, 2022Updated 3 years ago
- A DataFusion table provider for executing SQL on remote databases.☆14Updated this week
- HAL for the RI5CY core of RV32M1 heterogeneous (two ARM and two RISC-V) quad-core SoC.☆10Sep 28, 2020Updated 6 years ago
- Proposed API for type-driven member access☆13Feb 28, 2022Updated 4 years ago