A high-throughput and memory-efficient inference and serving engine for LLMs
☆10Jul 22, 2023Updated 3 years ago
Alternatives and similar repositories for vllm
Users that are interested in vllm are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Transformer related optimization, including BERT, GPT☆17Jul 29, 2023Updated 3 years ago
- CuteDSL Swizzle Visualize Tool☆14Feb 22, 2026Updated 8 months ago
- A curated list for Efficient Large Language Models☆11Mar 25, 2024Updated 2 years ago
- ☆15Feb 23, 2025Updated last year
- Optimize QWen1.5 models with TensorRT-LLM☆17May 14, 2024Updated 2 years ago
- GPUs on demand by Runpod - Special Offer Available • AdRun AI, ML, and HPC workloads on powerful cloud GPUs—without limits or wasted spend. Deploy GPUs in under a minute and pay by the second.
- ☆13Mar 28, 2025Updated last year
- 基于SpringCloud(Hoxton.SR7) + SpringBoot(2.2.9.RELEASE) 的SaaS型微服务快速开发平台,具有统一授权、认证后台管理系统,其中包含具备用户管理、资源权限管理、网关API支持白名单、分布式事务等多个模块,支持多业务系统并行开发…☆18Oct 8, 2020Updated 6 years ago
- A real-time, streaming audio engine for full-duplex speech interaction powered by AFE and LLM☆11Nov 10, 2026Updated last week
- ☆15Jun 26, 2024Updated 2 years ago
- ☆30Mar 27, 2023Updated 3 years ago
- 📚Some papers & books I’ve read.☆31May 7, 2026Updated 6 months ago
- ffmpeg+cuvid+tensorrt+multicamera☆12Dec 31, 2024Updated last year
- Category independent sequential image segmentation. Find a segment containing a given image pixel within a given attention mask using a …☆13Oct 11, 2021Updated 5 years ago
- ACEnet: Anatomical Context-Encoding Network for Neuroanatomy Segmentation☆16Feb 14, 2022Updated 4 years ago
- 1-Click AI Models by DigitalOcean Gradient • AdDeploy popular AI models on DigitalOcean Gradient GPU virtual machines with just a single click. Zero configuration with optimized deployments.
- ☆19Mar 4, 2025Updated last year
- a 3D FPN backbone for object detection☆15Nov 7, 2018Updated 8 years ago
- 基于MNN和cpp-httplib实现的简单llm推理服务☆10Apr 9, 2024Updated 2 years ago
- A Typescript API client for AUTOMATIC111/stable-diffusion-webui☆11Oct 18, 2023Updated 3 years ago
- 基于Vue+Element-ui+axios开发的一套电商平台前端模板☆21Oct 24, 2020Updated 6 years ago
- Simple and efficient memory pool is implemented with C++11.☆10Jun 2, 2022Updated 4 years ago
- Evaluation tool for object detection models☆15Jan 6, 2026Updated 10 months ago
- 异步命令组件☆21Aug 1, 2023Updated 3 years ago
- ☆13Mar 12, 2026Updated 8 months ago
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- Allows you to use a macro prefix_all to prefix every attribute in structs and enums on serialization☆12Mar 28, 2021Updated 5 years ago
- UNetMultiLane 多车道线和车道线类型识别rknn部署C++代码☆10Jul 4, 2024Updated 2 years ago
- MYSQL 监控工具,优化工具,各种工具为一体的java spring boot 项目☆26Jun 21, 2022Updated 4 years ago
- The reproduction of SRCNN method for super-resolution☆30Sep 5, 2019Updated 7 years ago
- A high-throughput and memory-efficient inference and serving engine for LLMs☆17Jun 3, 2024Updated 2 years ago
- An open implementation for Feishu Skills, allowing agents to send messages, write documents, managing TODOs for you.☆14Feb 28, 2026Updated 8 months ago
- An auxiliary project analysis of the characteristics of KV in DiT Attention.☆34Nov 29, 2024Updated last year
- envd Website☆12Oct 28, 2026Updated 3 weeks ago
- code for MM-WHS 2017: Multi-Modality Whole Heart Segmentation☆15Sep 1, 2017Updated 9 years ago
- GPU virtual machines on DigitalOcean Gradient AI • AdGet to production fast with high-performance AMD and NVIDIA GPUs you can spin up in seconds. The definition of operational simplicity.
- Send and receive OSC data to and from bevy and other programs or controllers☆11Jul 28, 2024Updated 2 years ago
- ☆13Sep 22, 2025Updated last year
- Top-2 Solution for CVPR UG2+ Track2☆18May 25, 2022Updated 4 years ago
- ☆62Nov 21, 2024Updated last year
- Extract frontmatter from markdown code blocks using remark, and do interesting things!☆13Dec 4, 2022Updated 3 years ago
- HAL for the RI5CY core of RV32M1 heterogeneous (two ARM and two RISC-V) quad-core SoC.☆10Sep 28, 2020Updated 6 years ago
- workflow of nndeploy☆14Nov 5, 2025Updated last year
Are these results useful?