LLM Inference with Deep Learning Accelerator.
☆59Jan 23, 2025Updated last year
Alternatives and similar repositories for LLM-Inference-Acceleration
Users that are interested in LLM-Inference-Acceleration are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- A scheduling framework for multitasking over diverse XPUs, including GPUs, NPUs, ASICs, and FPGAs☆14Jun 17, 2026Updated 4 months ago
- 大模型API性能指标比较 - 深入分析TTFT、TPS等关键指标☆20Sep 12, 2024Updated 2 years ago
- Major project - kannada LLM for farmers☆12May 20, 2024Updated 2 years ago
- ☆16Apr 10, 2023Updated 3 years ago
- Accompany repo for the PromptQuine paper (ICML 2025).☆16Jan 27, 2026Updated 9 months ago
- End-to-end encrypted cloud storage - Proton Drive • AdSpecial offer: 40% Off Yearly / 80% Off First Month. Protect your most important files, photos, and documents from prying eyes.
- [ICLR2026] Learnable Sparsity for Vision Generative Models☆14Mar 3, 2026Updated 7 months ago
- ☆15Jan 21, 2021Updated 5 years ago
- introduce AI infra knowledges. 人工智能系统基础架构知识库☆19Jun 4, 2023Updated 3 years ago
- A Claims-Annotated Catalog of LLM-Driven Kernel Generation Systems☆16Oct 20, 2026Updated 2 weeks ago
- ☆13Jun 20, 2023Updated 3 years ago
- Lumina is a user-friendly tool to test the correctness and performance of hardware network stacks.☆29Jan 8, 2024Updated 2 years ago
- Continuous Pipelined Speculative Decoding☆22May 25, 2026Updated 5 months ago
- ☆13Jul 24, 2024Updated 2 years ago
- Multiple GEMM operators are constructed with cutlass to support LLM inference.☆20Aug 3, 2025Updated last year
- GPU virtual machines on DigitalOcean Gradient AI • AdGet to production fast with high-performance AMD and NVIDIA GPUs you can spin up in seconds. The definition of operational simplicity.
- Since the emergence of chatGPT in 2022, the acceleration of Large Language Model has become increasingly important. Here is a list of pap…☆285Mar 6, 2025Updated last year
- PrefixKV: Adaptive Prefix KV Cache is What Vision Instruction-Following Models Need for Efficient Generation [NeurIPS 2025]☆20Oct 11, 2025Updated last year
- Vitis 部署加速器工作流介绍☆14Jan 10, 2025Updated last year
- A NCCL extension library, designed to efficiently offload GPU memory allocated by the NCCL communication library.☆122Dec 17, 2025Updated 10 months ago
- A Convolutional Neural Network Accelerator, which increases the process of convolution calculation. Based on Xilinx HLS design suite.☆12Jul 29, 2021Updated 5 years ago
- SGLang is a fast serving framework for large language models and vision language models.☆11Dec 15, 2025Updated 10 months ago
- GPU-accelerated LLM Training Simulator☆23Jun 26, 2025Updated last year
- 训练营讲义☆21Jan 21, 2025Updated last year
- Deferred Continuous Batching in Resource-Efficient Large Language Model Serving (EuroMLSys 2024)☆20May 28, 2024Updated 2 years ago
- AI Agents on DigitalOcean Gradient AI Platform • AdBuild production-ready AI agents using customizable tools or access multiple LLMs through a single endpoint. Create custom knowledge bases or connect external data.
- [FPGA'21] Microbenchmarks for Demystifying the Memory System of Modern Datacenter FPGAs for Software Programmers☆31Dec 16, 2021Updated 4 years ago
- A throughput-oriented high-performance serving framework for LLMs☆976Mar 29, 2026Updated 7 months ago
- [ICML 2025] Efficiently Serving Large Multimodal Models Using EPD Disaggregation☆28Jul 11, 2026Updated 3 months ago
- Adapt full attention LLM to sliding window attention☆14Feb 20, 2026Updated 8 months ago
- ☆19Sep 10, 2026Updated last month
- 📚A curated list of Awesome LLM/VLM Inference Papers with Codes: Flash-Attention, Paged-Attention, WINT8/4, Parallelism, etc.🎉☆5,542Oct 20, 2026Updated 2 weeks ago
- ☆14May 12, 2025Updated last year
- 吴恩达深度学习课程笔记PyTorch版☆21Nov 2, 2025Updated last year
- Some common algorithms and data structures implemented in the Scheme programming language.☆11Apr 27, 2022Updated 4 years ago
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- A simple script to plot the Roofline model for given HW platforms and applications☆10Mar 17, 2026Updated 7 months ago
- Self Reproduction Code of Paper "Reducing Transformer Key-Value Cache Size with Cross-Layer Attention (MIT CSAIL)☆17May 24, 2024Updated 2 years ago
- OnlineJudge项目(YuJudge)的判题服务器，支持docker一键部署☆11May 27, 2023Updated 3 years ago
- Official Github repository for the EuroSys '23 paper "FlexPass: A Case for Flexible Credit-based Transport for Datacenter Networks"☆10Mar 15, 2023Updated 3 years ago
- Cyclone Jet Rocket is a DDoS tool for System Security Technology course☆11Jun 5, 2017Updated 9 years ago
- CUDA-based Tree builder☆10May 12, 2024Updated 2 years ago
- Efficient and easy multi-instance LLM serving☆560Mar 12, 2026Updated 7 months ago