A GPT-2 inference engine written from scratch in CUDA and C++. Implements custom CUDA kernels for tiled matrix multiplication, LayerNorm, fused attention, transformer blocks, KV cache management, autoregressive token generation, and end-to-end GPT-2 inference with profiling and benchmarking.
☆44May 17, 2026Updated 6 months ago
Alternatives and similar repositories for gpt2-inference
Users that are interested in gpt2-inference are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Implemented and benchmarked KV cache compression methods for LLM inference in Triton. Optimized kernels for KIVI and TurboQuant with Flas…☆18Jun 19, 2026Updated 4 months ago
- 3d software renderer☆26Jul 30, 2026Updated 3 months ago
- Custom memory allocator in C++ built from scratch using mmap. Allocates a 1MB memory pool upfront and carves blocks from it to keep all a…☆38Apr 29, 2026Updated 6 months ago
- a collection of CUDA Kernels☆18Jul 24, 2026Updated 3 months ago
- A CUDA Python experiment demonstrating kernel fusion by combining ReLU and LayerNorm into a single GPU pass and comparing it against the …☆14Feb 28, 2026Updated 8 months ago
- GPUs on demand by Runpod - Special Offer Available • AdRun AI, ML, and HPC workloads on powerful cloud GPUs—without limits or wasted spend. Deploy GPUs in under a minute and pay by the second.
- Serving DeepSeek-V4-Flash-0731-NVFP4 on 2x RTX PRO 6000 Blackwell (sm_120): vLLM patches, tuned configs, serve scripts, benchmarks☆15Aug 25, 2026Updated 2 months ago
- a personal collection of my notes for ml sys☆137Oct 26, 2026Updated 3 weeks ago
- A custom, high-performance, disk-backed Approximate Nearest Neighbor (ANN) vector database built entirely from scratch in C. Feel free to…☆18Sep 19, 2026Updated 2 months ago
- fake CUTLASS to get peformance☆26Apr 28, 2026Updated 6 months ago
- ☆15Mar 11, 2026Updated 8 months ago
- A simple, thread safe & stop-the-world, Mark & Sweep Garbage Collector. Written in C++.☆10Feb 19, 2026Updated 8 months ago
- ☆10Apr 20, 2026Updated 6 months ago
- Execution-state orchestration layer for FlashRT: model-runtime adoption, stage-DAG scheduling, RTC action chunks, and capsule state contr…☆13Sep 7, 2026Updated 2 months ago
- Educational distributed training and inference library for heterogeneous local hardware: Mac minis, Raspberry Pis and GPUs over plain Pyt…☆91Oct 7, 2026Updated last month
- Virtual machines for every use case on DigitalOcean • AdGet dependable uptime with 99.99% SLA, simple security tools, and predictable monthly pricing with DigitalOcean's virtual machines, called Droplets.
- An SI pet for your Mac that does real tasks.☆19Oct 2, 2026Updated last month
- Resources to learn llvm-ripple programming☆12Nov 10, 2026Updated last week
- Publishing SDK that turns Markdown, MDX, and JSX into publication-grade technical articles on your own React/Next.js/Vite site.☆14Sep 21, 2026Updated last month
- ☆17Aug 24, 2026Updated 2 months ago
- event-sourced adk for go☆12May 7, 2026Updated 6 months ago
- Row-wise block scaling for fp8 quantization matrix multiplication. Solution to GPU mode AMD challenge.☆19Feb 9, 2026Updated 9 months ago
- A PyTorch implementation of the GPT-OSS-20B architecture. All components are coded from scratch: RoPE with YaRN, RMSNorm, SwiGLU with cla…☆239Dec 2, 2025Updated 11 months ago
- ☆46May 4, 2025Updated last year
- 1.3B world model running at 16 FPS on an RTX 5090☆18Oct 29, 2026Updated 2 weeks ago
- GPUs on demand by Runpod - Special Offer Available • AdRun AI, ML, and HPC workloads on powerful cloud GPUs—without limits or wasted spend. Deploy GPUs in under a minute and pay by the second.
- cpu inference for ternary neural nets. no deps. just c++ and simd.☆23Oct 18, 2026Updated last month
- Nintendo Entertainment System Emulator☆13Aug 10, 2026Updated 3 months ago
- WIP (out of tree) Rust implementation of TPC-DS generators.☆10Jul 2, 2026Updated 4 months ago
- Flappy Bird but with your head using computer vision☆12Jan 16, 2022Updated 4 years ago
- SysAct is a terminal‑user‑interface (TUI) application written in Go, built with the Bubble Tea framework and Lip Gloss styling. It provid…☆15Jun 1, 2025Updated last year
- Synchronize the team repository with the services we use☆20Mar 31, 2025Updated last year
- Minimal FlashAttention in CUDA C++/CuTe: readable WMMA/CuTe kernels, no NxN workspace, up to 4.5x faster than naive PyTorch☆30Jun 2, 2026Updated 5 months ago
- FlashRT-HF-kernels contains standalone FlashRT CUDA/CUTLASS kernels prepared for the Hugging Face kernels community, focusing on small-ba…☆27Aug 21, 2026Updated 2 months ago
- Internet never forgots and now thought police never fails☆14Jul 27, 2025Updated last year
- GPU virtual machines on DigitalOcean Gradient AI • AdGet to production fast with high-performance AMD and NVIDIA GPUs you can spin up in seconds. The definition of operational simplicity.
- An expense tracking application that embodiesminimalist design philosophy with clean typography, subtle shadows, and elegant micro-intera…☆15Aug 31, 2025Updated last year
- ✨ CLI Animation Library for your NodeJS Projects☆20Feb 14, 2026Updated 9 months ago
- A modular AI agent framework with secure CLI and toolkit architecture☆14Sep 19, 2025Updated last year
- 🦗 Belalang Language☆18Oct 17, 2026Updated last month
- CMU 15-418/15-618 Parallel Computer Architecture and Programming☆11Oct 23, 2023Updated 3 years ago
- IMMA-based **FP8-as-storage** GEMM experiments for Ampere (sm_86 / RTX 3090 Ti).☆28Jan 30, 2026Updated 9 months ago
- ☆11Nov 16, 2025Updated last year
Are these results useful?