A GPT-2 inference engine written from scratch in CUDA and C++. Implements custom CUDA kernels for tiled matrix multiplication, LayerNorm, fused attention, transformer blocks, KV cache management, autoregressive token generation, and end-to-end GPT-2 inference with profiling and benchmarking.
☆44May 17, 2026Updated 5 months ago
Alternatives and similar repositories for gpt2-inference
Users that are interested in gpt2-inference are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Custom memory allocator in C++ built from scratch using mmap. Allocates a 1MB memory pool upfront and carves blocks from it to keep all a…☆38Apr 29, 2026Updated 6 months ago
- Implemented and benchmarked KV cache compression methods for LLM inference in Triton. Optimized kernels for KIVI and TurboQuant with Flas…☆18Jun 19, 2026Updated 4 months ago
- a collection of CUDA Kernels☆18Jul 24, 2026Updated 3 months ago
- A custom, high-performance, disk-backed Approximate Nearest Neighbor (ANN) vector database built entirely from scratch in C. Feel free to…☆18Sep 19, 2026Updated last month
- 3d software renderer☆26Jul 30, 2026Updated 3 months ago
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- a personal collection of my notes for ml sys☆126Updated this week
- Fused MoE kernel☆23Oct 23, 2026Updated last week
- Flash-Flash KDA: H100-optimized Flash Kimi Delta Attention kernels☆24Jul 24, 2026Updated 3 months ago
- fake CUTLASS to get peformance☆26Apr 28, 2026Updated 6 months ago
- Flash Attention from scratch, tiled CUDA forward kernel, online softmax with running max and correction factor, recomputation trick in ba…☆19Mar 6, 2026Updated 7 months ago
- ☆15Mar 11, 2026Updated 7 months ago
- A simple, thread safe & stop-the-world, Mark & Sweep Garbage Collector. Written in C++.☆10Feb 19, 2026Updated 8 months ago
- event-sourced adk for go☆12May 7, 2026Updated 5 months ago
- ☆10Apr 20, 2026Updated 6 months ago
- Serverless GPU API endpoints on Runpod - Get Bonus Credits • AdSkip the infrastructure headaches. Auto-scaling, pay-as-you-go, no-ops approach lets you focus on innovating your application.
- Parallel Programming Course☆10Oct 23, 2026Updated last week
- An SI pet for your Mac that does real tasks.☆19Oct 2, 2026Updated 3 weeks ago
- In-process, multi master, distributed database☆21Jun 18, 2026Updated 4 months ago
- How to Write to SSDs (VLDB 2026, Best Research Paper Honorable Mention Award), out-of-place write LeanStore☆28Sep 28, 2026Updated last month
- 1.3B world model running at 16 FPS on an RTX 5090☆16Oct 7, 2026Updated 3 weeks ago
- GPU-accelerated particle life simulation in Rust using wgpu☆11Jul 27, 2026Updated 3 months ago
- The R8 fantasy retro CPU architecture.☆15Updated this week
- Row-wise block scaling for fp8 quantization matrix multiplication. Solution to GPU mode AMD challenge.☆19Feb 9, 2026Updated 8 months ago
- FlashRT-HF-kernels contains standalone FlashRT CUDA/CUTLASS kernels prepared for the Hugging Face kernels community, focusing on small-ba…☆25Aug 21, 2026Updated 2 months ago
- Virtual machines for every use case on DigitalOcean • AdGet dependable uptime with 99.99% SLA, simple security tools, and predictable monthly pricing with DigitalOcean's virtual machines, called Droplets.
- cpu inference for ternary neural nets. no deps. just c++ and simd.☆23Oct 18, 2026Updated last week
- A PyTorch implementation of the GPT-OSS-20B architecture. All components are coded from scratch: RoPE with YaRN, RMSNorm, SwiGLU with cla…☆238Dec 2, 2025Updated 10 months ago
- ☆45May 4, 2025Updated last year
- Internet never forgots and now thought police never fails☆14Jul 27, 2025Updated last year
- An async processing payment pipeline in which users can withdraw there balance from there profiles. The project implements append only le…☆22May 18, 2026Updated 5 months ago
- Lindenmayer Systems is a parallel rewriting system, namely a variant of a formal grammar, most famously used to model the growth processe…☆12Mar 27, 2014Updated 12 years ago
- SysAct is a terminal‑user‑interface (TUI) application written in Go, built with the Bubble Tea framework and Lip Gloss styling. It provid…☆15Jun 1, 2025Updated last year
- Static site deployment orchestrator with Docker builds, MinIO artifact storage, and deployment status/log APIs.☆17Aug 15, 2026Updated 2 months ago
- ✨ CLI Animation Library for your NodeJS Projects☆20Feb 14, 2026Updated 8 months ago
- End-to-end encrypted email - Proton Mail • AdSpecial offer: 40% Off Yearly / 80% Off First Month. All Proton services are open source and independently audited for security.
- Qwen3.6/3.8-27B on one RTX 5090: 576K pooled context across 12 sessions at FP8-level quality from 5.75 bpw. EXL3 trellis weights, 4-bit t…☆28Oct 23, 2026Updated last week
- 🃏 customizable badges for the web☆14Aug 16, 2026Updated 2 months ago
- CMU 15-418/15-618 Parallel Computer Architecture and Programming☆11Oct 23, 2023Updated 3 years ago
- IMMA-based **FP8-as-storage** GEMM experiments for Ampere (sm_86 / RTX 3090 Ti).☆28Jan 30, 2026Updated 9 months ago
- Compiler for the teal programming language (typed superset of Lua)☆19Mar 12, 2025Updated last year
- Gray-Scott reaction-diffusion system in 3D using CUDA☆13Jun 8, 2019Updated 7 years ago
- LumenPage is a Canvas editor monorepo focused on paginated document editing Replace Tiptap's view implementation with Canvas.☆16Apr 14, 2026Updated 6 months ago