A high-throughput and memory-efficient inference and serving engine for LLMs
☆41Aug 13, 2026Updated last month
Alternatives and similar repositories for vllm
Users that are interested in vllm are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- ☆74Feb 27, 2026Updated 7 months ago
- Mixed-precision quantization for LLMs. Every layer refracts into a different format based on its sensitivity. Native compressed-tensors e…☆103Updated this week
- antirez/ds4-style hybrid quant DeepSeek V4 Flash on a single DGX Spark via vLLM☆17May 11, 2026Updated 5 months ago
- Official code for paper "Revisiting Model Interpolation for Efficient Reasoning"☆17Jul 14, 2026Updated 2 months ago
- ☆16Jun 3, 2026Updated 4 months ago
- Wordpress hosting with auto-scaling - Free Trial Offer • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- Source code for MA4270: Data Modelling and Computation on Transformers and Nadaraya-Watson Kernel Regression☆19May 29, 2024Updated 2 years ago
- RTX 6000 Pro Wiki — Running Large LLMs (Qwen3.5-397B, Kimi-K2.5, GLM-5) on PCIe GPUs without NVLink☆1,131Updated this week
- ☆42Jul 13, 2026Updated 2 months ago
- FlashQLA TileLang GDN kernels ported to NVIDIA Blackwell consumer (GB10 / DGX Spark)☆18Jun 5, 2026Updated 4 months ago
- vLLM fork with Marlin W4A8 SM121 patches + TMA module☆15Mar 23, 2026Updated 6 months ago
- Mixed-precision numerics benchmarks in Rust and Python - covering GEMMs, SYRKs, DOTs, and higher-level BLAS and LAPACK-style functionalit…☆33Oct 1, 2026Updated last week
- Dockerfiles for poetry/mlc-llm(rk3588)/...☆10Sep 13, 2023Updated 3 years ago
- 一款被动扫描ssrf的burpsuite插件☆20Dec 30, 2022Updated 3 years ago
- Archived: b12x development has moved to FlashInfer. See README for migration and PR-porting guidance.☆271Updated this week
- 1-Click AI Models by DigitalOcean Gradient • AdDeploy popular AI models on DigitalOcean Gradient GPU virtual machines with just a single click. Zero configuration with optimized deployments.
- wubuwizard: The best inference engine☆28Aug 27, 2026Updated last month
- DeepSeek-V4-Flash on Ampere SM 8.6 via vLLM (pyref kernel replacements)☆35Jul 2, 2026Updated 3 months ago
- Code showing how to use a model based on the ML model base class.☆10Sep 30, 2022Updated 4 years ago
- ☆120Aug 6, 2026Updated 2 months ago
- Docker configuration for running VLLM on dual DGX Sparks☆2,378Updated this week
- Triton for AMD MI25/50/60. Development repository for the Triton language and compiler☆36Dec 15, 2025Updated 9 months ago
- A compute framework for building Search, RAG, Recommendations and Analytics over complex (structured+unstructured) data, with ultra-modal…☆12Sep 16, 2024Updated 2 years ago
- ☆14Oct 22, 2023Updated 2 years ago
- ☆34Sep 3, 2026Updated last month
- Managed Database hosting by DigitalOcean • AdPostgreSQL, MySQL, MongoDB, Kafka, Valkey, and OpenSearch available. Automatically scale up storage and focus on building your apps.
- Features and labels engineering of raw data of quotes of several stocks.☆32Oct 9, 2019Updated 7 years ago
- vllm混合推理扩展插件，支持多NUMA混合推理，单卡推理Qwen3-Next模型可达1000+ prefill☆34Nov 7, 2025Updated 11 months ago
- Kubeflow on OpenShift☆14Jan 24, 2019Updated 7 years ago
- Some benchmark results of small models and quants that fit on DGX Spark☆51Aug 23, 2026Updated last month
- Experimentation on google's gemma model☆15Mar 6, 2024Updated 2 years ago
- This repo consists of code for plotting top loss images☆13May 18, 2020Updated 6 years ago
- OpenAI-compatible server with a live view into any HF model's residual stream. pip install brainscope☆52Sep 23, 2026Updated 2 weeks ago
- PydanticAI开源框架，搭建基于PostgreSQL、MySQL的Text2SQL应用进行SQL语句生成，支持GPT大模型、国产大模型、开源本地大模型☆18Dec 26, 2024Updated last year
- ☆84Oct 4, 2026Updated last week
- Wordpress hosting with auto-scaling - Free Trial Offer • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- Implementation of <Model Merging with Functional Dual Anchors>☆47Nov 23, 2025Updated 10 months ago
- A packer which adds encrypted shell to protect your PE file☆29Nov 16, 2024Updated last year
- This repository hosts community contributed Kestrel analytics☆19May 28, 2024Updated 2 years ago
- 对任意http网站添加指定漏洞 目前实现部分漏洞的追加 支持本身http服务 支持反向代理追加☆28Oct 8, 2022Updated 4 years ago
- Indexed regex search for large codebases, powered by trigram / sparse n‑gram indexes. A grep-like CLI that builds a local on-disk index, …☆47Apr 30, 2026Updated 5 months ago
- ☆12Jun 17, 2023Updated 3 years ago
- Build a Full stack Q&A Chatbot with Langchain, and LLM Models on Amazon Sagemaker☆12Nov 10, 2023Updated 2 years ago