A GPT-2 inference engine written from scratch in CUDA and C++. Implements custom CUDA kernels for tiled matrix multiplication, LayerNorm, fused attention, transformer blocks, KV cache management, autoregressive token generation, and end-to-end GPT-2 inference with profiling and benchmarking.
☆44May 17, 2026Updated 3 months ago
Alternatives and similar repositories for gpt2-inference
Users that are interested in gpt2-inference are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- a personal collection of my notes for ml sys☆124Sep 1, 2026Updated 2 weeks ago
- Flash Attention from scratch, tiled CUDA forward kernel, online softmax with running max and correction factor, recomputation trick in ba…☆19Mar 6, 2026Updated 6 months ago
- ☆43Sep 5, 2026Updated last week
- How to Write to SSDs (VLDB 2026), out-of-place write LeanStore☆25Aug 16, 2026Updated 3 weeks ago
- In-process, multi master, distributed database☆21Jun 18, 2026Updated 2 months ago
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- Row-wise block scaling for fp8 quantization matrix multiplication. Solution to GPU mode AMD challenge.☆19Feb 9, 2026Updated 7 months ago
- Daisytuner Optimizing Compiler Collection (docc)☆22Updated this week
- A PyTorch implementation of the GPT-OSS-20B architecture. All components are coded from scratch: RoPE with YaRN, RMSNorm, SwiGLU with cla…☆238Dec 2, 2025Updated 9 months ago
- ☆28Apr 14, 2026Updated 5 months ago
- MLIR dialect for libgccjit☆24Dec 3, 2024Updated last year
- ☆11Updated this week
- Animation of Yves Lafont's Interaction Combinators☆24Jan 30, 2022Updated 4 years ago
- This comprehensive learning repository is designed to transform software engineers into expert AI kernel developers, focusing on the cutt…☆73Mar 25, 2026Updated 5 months ago
- 自动识别文本中的关键词并加粗处理。☆10Oct 30, 2024Updated last year
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- Pure Triton kernels for Qwen3.5-27B inference on NVIDIA B200☆123Feb 28, 2026Updated 6 months ago
- multi-master-paxos with 3 nodes☆14Apr 11, 2022Updated 4 years ago
- An experimental optimizing compiler written by LLM☆28May 23, 2026Updated 3 months ago
- Andrej Kapathy's micrograd implemented in c☆30Aug 7, 2024Updated 2 years ago
- Repository for the CUDA H100 Course☆78Apr 12, 2026Updated 5 months ago
- My submission for the GPUMODE/AMD fp8 mm challenge☆29Jun 4, 2025Updated last year
- Fast semantic search for biorXiv manuscripts☆12Feb 16, 2025Updated last year
- Building your own docker like container by using only linux bash commands☆16Oct 17, 2025Updated 10 months ago
- 一个手把手教你从零开始编写GPT并训练大语言模型的教程☆102Jan 20, 2025Updated last year
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- ☆24Jul 6, 2026Updated 2 months ago
- Conway's Game Of Life implemented using MPI & OpenMP☆13Mar 5, 2019Updated 7 years ago
- ☆12Aug 30, 2024Updated 2 years ago
- Reverse engineering notes. Personal reference only. Everything here is a best-guess reconstruction.☆115Aug 11, 2026Updated last month
- A series of high-performance GEMM (General Matrix Multiply) implementations Iteratively optimised for H100 GPUs in Pure CUDA.☆82Feb 18, 2026Updated 6 months ago
- A VSCode extension that generating directory structure for the current project by copy and paste.☆20Feb 14, 2026Updated 7 months ago
- ☆21May 11, 2026Updated 4 months ago
- A end-to-end MLOps pipeline for predicting telecom customer churn, featuring automated data preprocessing, ML model training, experiment …☆17Oct 20, 2025Updated 10 months ago
- Splice and merge videos from the terminal☆25Oct 4, 2025Updated 11 months ago
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- Watch Shows/Movies with Tracking and resume with Google content links☆17Sep 10, 2025Updated last year
- ☆45May 4, 2025Updated last year
- A Modern Terminal-Based Network Monitor☆18Oct 23, 2025Updated 10 months ago
- Inverted triple Pendulum☆18May 13, 2019Updated 7 years ago
- ☆13Oct 20, 2021Updated 4 years ago
- SlimArray compresses uint32 into several bits, by using a polynomial to describe overall trend of an array.☆51May 30, 2022Updated 4 years ago
- NVidia sass disassembler/inline patcher☆105Updated this week