通过实验对比LLM推理中Prefill和Decoding阶段的吞吐量差异，揭示性能瓶颈，解释PD分离优化技术的原理。包含CUDA和Apple MPS (M系列芯片) 的测试脚本。
☆23May 22, 2025Updated last year
Alternatives and similar repositories for LLM-Prefill-Decode-Benchmark
Users that are interested in LLM-Prefill-Decode-Benchmark are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- LaTeX Examples Document Source☆12Apr 9, 2024Updated 2 years ago
- This repository contains papers for a comprehensive survey on accelerated generation techniques in Large Language Models (LLMs).☆11May 24, 2024Updated 2 years ago
- convert pdf file to images☆14Oct 30, 2017Updated 8 years ago
- [AAAI 2026] AutoTool: Efficient Tool Selection for Large Language Model Agents☆47Dec 28, 2025Updated 9 months ago
- The smallest 6502 / NES CPU simulator in JS (944b gzipped)☆15Mar 1, 2022Updated 4 years ago
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- ROCm Driver RDMA Peer to Peer Support☆22Mar 21, 2019Updated 7 years ago
- An Empirical Study of Memorization in NLP (ACL 2022)☆13Jun 22, 2022Updated 4 years ago
- demonstration for our ACL 2018 paper, "On the Practical Computational Power of Finite Precision RNNs for Language Recognition"☆11May 26, 2019Updated 7 years ago
- Distributed SDDMM Kernel☆13Jul 8, 2022Updated 4 years ago
- Optimised x86-64 gzip decompressor☆37Feb 26, 2018Updated 8 years ago
- llvmのAZ Processor Backend☆11Oct 29, 2013Updated 12 years ago
- 8-bit RISC Processor on Logisim☆14Oct 1, 2020Updated 6 years ago
- ☆21Apr 18, 2024Updated 2 years ago
- Minimal FPGA Processor Core for Stack-based CPU for CPLDs Using Bit-Serial Architecture☆18Sep 6, 2013Updated 13 years ago
- 1-Click AI Models by DigitalOcean Gradient • AdDeploy popular AI models on DigitalOcean Gradient GPU virtual machines with just a single click. Zero configuration with optimized deployments.
- CUDA C simple application for Nvidia's GPU☆11Jun 7, 2022Updated 4 years ago
- ☆10Feb 8, 2023Updated 3 years ago
- A small 6502 system build on a Lattice Icestick FPGA development board☆16Jun 24, 2019Updated 7 years ago
- Source code and data for the EDM 2022 paper☆12May 16, 2022Updated 4 years ago
- Some microbenchmarks and design docs before commencement☆11Feb 1, 2021Updated 5 years ago
- ☆12Jun 3, 2019Updated 7 years ago
- ☆16Aug 19, 2024Updated 2 years ago
- Example of applying CUDA graphs to LLaMA-v2☆11Aug 25, 2023Updated 3 years ago
- ☆13Jan 7, 2025Updated last year
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- ☆15Mar 11, 2026Updated 7 months ago
- An Efficient and Versatile Inference Engine for Distributed LLM Serving☆74Updated this week
- ☆46Oct 11, 2025Updated last year
- A method for estimating causal effects in time-series data. Uses available data to automatically find natural experiments for identifying…☆17Dec 16, 2019Updated 6 years ago
- Tasks and tutorials using Graphore's IPU with Hugging Face. Originally at https://github.com/gradient-ai/Graphcore-HuggingFace☆16Mar 12, 2024Updated 2 years ago
- LLM inference optimization simulator☆13Jul 12, 2025Updated last year
- Inline PTX Assembly in CUDA example☆15Oct 5, 2026Updated last week
- Code for NAACL 2019 paper: Distant Supervision Relation Extraction with Intra-Bag and Inter-Bag Attentions☆115Jul 20, 2021Updated 5 years ago
- Advanced library for packet processing. Includes NetVM, NetPDL, and NetPFL.☆21May 15, 2019Updated 7 years ago
- Virtual machines for every use case on DigitalOcean • AdGet dependable uptime with 99.99% SLA, simple security tools, and predictable monthly pricing with DigitalOcean's virtual machines, called Droplets.
- Prolog interpreter with support for weak unification. Fork of https://bitbucket.org/cfbolz/pyrolog/☆15Jun 23, 2020Updated 6 years ago
- A simple tool for parsing the profile.json file of mxnet☆14Aug 1, 2018Updated 8 years ago
- OK:iCE40Pro, based on iCE40up5k FPGA Core, is an Open Source Educational FPGA GamePad Console.☆16May 6, 2023Updated 3 years ago
- A list of research resources that I've appreciated.☆12Dec 10, 2019Updated 6 years ago
- 🔥Keywords and URLs Censored on the Chinese Internet☆13Feb 22, 2020Updated 6 years ago
- Deep Learning model to tackle the Fake News Challenge☆13Nov 6, 2018Updated 7 years ago
- Simple starter CMake project that uses NVBench.☆15May 6, 2025Updated last year