A GPT-2 inference engine written from scratch in CUDA and C++. Implements custom CUDA kernels for tiled matrix multiplication, LayerNorm, fused attention, transformer blocks, KV cache management, autoregressive token generation, and end-to-end GPT-2 inference with profiling and benchmarking.
☆40May 17, 2026Updated last month
Alternatives and similar repositories for gpt2-inference
Users that are interested in gpt2-inference are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Row-wise block scaling for fp8 quantization matrix multiplication. Solution to GPU mode AMD challenge.☆19Feb 9, 2026Updated 5 months ago
- a personal collection of my notes for ml sys☆101Updated this week
- out-of-place write LeanStore (VLDB 2026)☆20Jun 8, 2026Updated last month
- MLIR dialect for libgccjit☆24Dec 3, 2024Updated last year
- My submission for the GPUMODE/AMD fp8 mm challenge☆29Jun 4, 2025Updated last year
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- Animation of Yves Lafont's Interaction Combinators☆24Jan 30, 2022Updated 4 years ago
- Repository for the CUDA H100 Course☆66Apr 12, 2026Updated 3 months ago
- 自动识别文本中的关键词并加粗处理。☆10Oct 30, 2024Updated last year
- Reverse engineering notes. Personal reference only. Everything here is a best-guess reconstruction.☆63Jun 23, 2026Updated 3 weeks ago
- Andrej Kapathy's micrograd implemented in c☆30Aug 7, 2024Updated last year
- Hook syscalls from ring0 without triggering PatchGuard☆24Oct 29, 2025Updated 8 months ago
- ☆39Updated this week
- An FPGA design for simulating biological neurons☆19Jul 5, 2024Updated 2 years ago
- ☆13Oct 20, 2021Updated 4 years ago
- Deploy open-source AI quickly and easily - Special Bonus Offer • AdRunpod Hub is built for open source. One-click deployment and autoscaling endpoints without provisioning your own infrastructure.
- GEMM☆10Aug 26, 2023Updated 2 years ago
- Tutorial of OpenGL ES using PowerVR framework☆12Jan 4, 2023Updated 3 years ago
- High-performance GPU kernels for LLM inference in OpenAI Triton. Fused RMSNorm, SwiGLU, INT8 GEMM with benchmarks and roofline analysis.☆31Apr 5, 2026Updated 3 months ago
- ☆14Aug 15, 2022Updated 3 years ago
- Region-level profiling for CUDA kernels with trace, NVBit, CUPTI, NSys, and an interactive Explorer.☆120Apr 17, 2026Updated 2 months ago
- A handy plugin for copying requests/responses directly from Burp, some extra magic included.☆13Oct 15, 2021Updated 4 years ago
- coffeescript based hardware description language☆14Jan 14, 2022Updated 4 years ago
- Minimalistic, hackable PyTorch implementation of SimSiam in ~400 lines. Achieves good performance on ImageNet with ResNet50. Features dis…☆22Nov 25, 2024Updated last year
- ☆11May 16, 2026Updated last month
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- Repository for VSD-IAT Workshop: Physical Verification using SKY130☆10Aug 15, 2021Updated 4 years ago
- Lets build a Deep Learning Framework!☆27Mar 12, 2026Updated 4 months ago
- Efficient kernel for RMS normalization with fused operations, includes both forward and backward passes, compatibility with PyTorch.☆13Jun 5, 2024Updated 2 years ago
- easy-phi☆25Feb 4, 2015Updated 11 years ago
- ☆13Aug 22, 2022Updated 3 years ago
- An MLIR-based compiler that takes GPU kernels and compiles them to real hardware instructions. Interactive web visualizer included.☆138Mar 21, 2026Updated 3 months ago
- ☆49Dec 11, 2020Updated 5 years ago
- An implementation of the Llama architecture, to instruct and delight☆21May 31, 2025Updated last year
- 🎓Automatically Update circult-eda-mlsys-tinyml Papers Daily using Github Actions (Update Every 8th hours)☆10Updated this week
- AI Agents on DigitalOcean Gradient AI Platform • AdBuild production-ready AI agents using customizable tools or access multiple LLMs through a single endpoint. Create custom knowledge bases or connect external data.
- -☆11Nov 21, 2020Updated 5 years ago
- My tests and experiments with some popular dl frameworks.☆17Sep 11, 2025Updated 10 months ago
- Copy millions of objects in minutes☆12Oct 21, 2019Updated 6 years ago
- GEMV implementation with CUTLASS☆21Aug 21, 2025Updated 10 months ago
- Example for PagedOut!☆25Oct 22, 2019Updated 6 years ago
- ☆14Nov 3, 2025Updated 8 months ago
- 《汇编语言一发入魂》配套代码☆15May 30, 2020Updated 6 years ago