Analyze LLM inference: FLOPs, memory, Roofline model. Supports GQA, MoE, MLA, RoPE, SwiGLU. 19 models × 20+ hardware platforms.
☆22Apr 16, 2026Updated 4 months ago
Alternatives and similar repositories for LLM-para
Users that are interested in LLM-para are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- ☆19Apr 15, 2025Updated last year
- Verilog-Based-NoC-Simulator☆12May 4, 2016Updated 10 years ago
- H2-LLM: Hardware-Dataflow Co-Exploration for Heterogeneous Hybrid-Bonding-based Low-Batch LLM Inference☆116Apr 26, 2025Updated last year
- ☆42Jun 25, 2026Updated 2 months ago
- Instruction Pointer Classifier and Dynamic Degree Stream based Hardware Cache Prefetching☆16Nov 16, 2019Updated 6 years ago
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- CoralNPU behavior simulator based on MPACT-Sim☆27Jun 10, 2026Updated 2 months ago
- A skill for automatically optimizing CUDA code.☆43Mar 26, 2026Updated 5 months ago
- ☆17Mar 29, 2026Updated 5 months ago
- Official codebase for the MLSys 2026 paper "IntAttention: A Fully Integer Attention Pipeline for Efficient Edge Inference". It enables hi…☆21May 29, 2026Updated 3 months ago
- 训练营讲义☆21Jan 21, 2025Updated last year
- InstAttention: In-Storage Attention Offloading for Cost-Effective Long-Context LLM Inference☆18Mar 30, 2025Updated last year
- Algorithm-System Co-design: accurate and efficient 2-bit KV cache quantization for LLM Inference.☆20May 20, 2026Updated 3 months ago
- ☆12Mar 26, 2024Updated 2 years ago
- [ASPLOS'26] HILOS: A Cost-Effective Near-Storage Processing Solution for Offline Inference of Long-Context LLMs☆22Jan 18, 2026Updated 7 months ago
- 1-Click AI Models by DigitalOcean Gradient • AdDeploy popular AI models on DigitalOcean Gradient GPU virtual machines with just a single click. Zero configuration with optimized deployments.
- ☆14Dec 20, 2024Updated last year
- ☆15Jan 21, 2023Updated 3 years ago
- Orchestrating many small GPU clusters for running serverless GPU workloads☆18Mar 15, 2026Updated 5 months ago
- ☆18May 28, 2024Updated 2 years ago
- An accelerator for high-dimensional approximate nearest neighbor search☆15May 17, 2025Updated last year
- AI Hedge Fund Repo integrate with DeepSeek V3 and R1 hosted on SiliconFlow.☆12Feb 3, 2025Updated last year
- Source code of the paper "OpSparse: a Highly Optimized Framework for Sparse General Matrix Multiplication on GPUs"☆16Aug 23, 2022Updated 4 years ago
- original 8bit CPU of ICF3-Z☆12Feb 20, 2020Updated 6 years ago
- ☆15Oct 7, 2021Updated 4 years ago
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- ☆124Nov 17, 2023Updated 2 years ago
- [ICML 2025] CommVQ: Commutative Vector Quantization for KV Cache Compression☆28Sep 2, 2025Updated 11 months ago
- ☆28Aug 18, 2019Updated 7 years ago
- vLLM Router☆58Mar 11, 2024Updated 2 years ago
- Tender: Accelerating Large Language Models via Tensor Decompostion and Runtime Requantization (ISCA'24)☆34Jul 4, 2024Updated 2 years ago
- The smallest 6502 / NES CPU simulator in JS (944b gzipped)☆15Mar 1, 2022Updated 4 years ago
- Simple example of how to write an Implicit GEMM Convolution in CUDA using the tensor core WMMA API and bindings for PyTorch.☆19Jun 29, 2023Updated 3 years ago
- 🔥 [ICML'26] ParisKV: Fast and Drift-Robust KV-Cache Retrieval for Long-Context LLMs☆34Aug 14, 2026Updated 2 weeks ago
- NVMe→VRAM weight streaming runtime for LLM inference using GPUDirect Storage☆25Feb 26, 2026Updated 6 months ago
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- ☆15Sep 30, 2023Updated 2 years ago
- Repo for SpecEE: Accelerating Large Language Model Inference with Speculative Early Exiting (ISCA25)☆75Apr 25, 2025Updated last year
- [Mlsys'22] Understanding gnn computational graph: A coordinated computation, io, and memory perspective☆22Sep 11, 2023Updated 2 years ago
- 3D-FPIM: An Extreme Energy-Efficient DNN Acceleration System Using 3D NAND Flash-Based In-Situ PIM Unit (MICRO 2022)☆29May 19, 2023Updated 3 years ago
- 8-bit RISC Processor on Logisim☆14Oct 1, 2020Updated 5 years ago
- NeuPIMs: NPU-PIM Heterogeneous Acceleration for Batched LLM Inferencing☆128Jun 19, 2024Updated 2 years ago
- Article: GPU-accelerated Proximity Graph Approximate Nearest Neighbor Search and Construction by Authors Yuanhang Yu, Dong Wen, Ying Zhan…☆24Jun 20, 2025Updated last year