โ255Nov 18, 2026Updated this week
Alternatives and similar repositories for Awesome-LLM-Inference-Engine
Users that are interested in Awesome-LLM-Inference-Engine are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Summary of some awesome work for optimizing LLM inferenceโ278Feb 14, 2026Updated 9 months ago
- ๐A curated list of Awesome LLM/VLM Inference Papers with Codes: Flash-Attention, Paged-Attention, WINT8/4, Parallelism, etc.๐โ5,564Oct 20, 2026Updated last month
- PowerNPU: Fast On-device LLM Inference with NPUs [ASPLOSโ25]โ10Oct 18, 2024Updated 2 years ago
- โ54Apr 29, 2025Updated last year
- A scheduling framework for multitasking over diverse XPUs, including GPUs, NPUs, ASICs, and FPGAsโ14Jun 17, 2026Updated 5 months ago
- Wordpress hosting with auto-scaling - Free Trial Offer โข AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- โ195Updated this week
- FlashInfer: Kernel Library for LLM Servingโ6,973Updated this week
- Dynamic Memory Management for Serving LLMs without PagedAttentionโ528Aug 24, 2026Updated 2 months ago
- A platform that provides users with easy access to AI services developed by Montimage and usage of explainable AI techniques (e.g., LIME,โฆโ10Feb 17, 2026Updated 9 months ago
- โ54Jul 12, 2026Updated 4 months ago
- A curated list of recent papers on efficient video attention for video diffusion models, including sparsification, quantization, and cachโฆโ69Oct 27, 2025Updated last year
- Nano vLLMโ16,512Apr 26, 2026Updated 6 months ago
- Large Language Model (LLM) Systems Paper Listโ2,332Oct 20, 2026Updated last month
- [ICML 2025 Spotlight] ShadowKV: KV Cache in Shadows for High-Throughput Long-Context LLM Inferenceโ316May 1, 2025Updated last year
- End-to-end encrypted cloud storage - Proton Drive โข AdSpecial offer: 40% Off Yearly / 80% Off First Month. Protect your most important files, photos, and documents from prying eyes.
- SGLang is a high-performance serving framework for large language models and multimodal models.โ38,882Updated this week
- (ACL2025 oral) SCOPE: Optimizing KV Cache Compression in Long-context Generationโ37May 28, 2025Updated last year
- Disaggregated serving system for Large Language Models (LLMs).โ847Apr 6, 2025Updated last year
- ไฝฟ็จ cutlass ๅฎ็ฐ flash-attention ็ฒพ็ฎ็๏ผๅ ทๆๆๅญฆๆไนโ59Aug 12, 2024Updated 2 years ago
- โ75Updated this week
- a curated list of high-quality papers on resource-efficient LLMs ๐ฑโ167Updated this week
- [ICLR'25] R-Sparse: Rank-Aware Activation Sparsity for Efficient LLM Inferenceโ21Apr 28, 2025Updated last year
- State-of-the-art LLM compression, built for production inference with vLLMโ3,979Updated this week
- A low-latency & high-throughput serving engine for LLMsโ539Updated this week
- Bare Metal GPUs on DigitalOcean Gradient AI โข AdPurpose-built for serious AI teams training foundational models, running large-scale inference, and pushing the boundaries of what's possible.
- PyTorch code for our paper "Progressive Binarization with Semi-Structured Pruning for LLMs"โ12Jul 11, 2026Updated 4 months ago
- An Open-source book with Modern CUDA learning Notes for Beginners - BF16/FP8/FP4, HGEMM, CuTe, Flash-Attention, etc.โ12,205Updated this week
- โ24Mar 7, 2025Updated last year
- FlexFlow Serve: Low-Latency, High-Performance LLM Servingโ91Sep 15, 2025Updated last year
- Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.โ7,090Updated this week
- My learning notes for ML SYS.โ7,695Oct 21, 2026Updated last month
- โ18Jun 3, 2025Updated last year
- A Comprehensive Survey on Long Context Language Modelingโ258Oct 24, 2026Updated 3 weeks ago
- LightLLM is a Python-based LLM (Large Language Model) inference and serving framework, notable for its lightweight design, easy scalabiliโฆโ4,368Updated this week
- 1-Click AI Models by DigitalOcean Gradient โข AdDeploy popular AI models on DigitalOcean Gradient GPU virtual machines with just a single click. Zero configuration with optimized deployments.
- CLIPO: Contrastive Learning in Policy Optimization Generalizes RLVRโ22Apr 7, 2026Updated 7 months ago
- โ38Updated this week
- Artifact of Chimeraโ18May 6, 2025Updated last year
- A GPU Virtualisation benchmarking tool - For LLM like workloads, overhead evaluations and on resource isolation metricsโ11Dec 30, 2025Updated 10 months ago
- [MLSys'25] QServe: W4A8KV4 Quantization and System Co-design for Efficient LLM Serving; [MLSys'25] LServe: Efficient Long-sequence LLM Seโฆโ868Mar 6, 2025Updated last year
- State of the art inference performance with modern accelerators on Kubernetesโ5,320Updated this week
- ๐ฐ Must-read papers and blogs on Speculative Decoding โก๏ธโ1,309Jun 27, 2026Updated 4 months ago
Are these results useful?