Flash Attention from scratch, tiled CUDA forward kernel, online softmax with running max and correction factor, recomputation trick in backward, O(N) memory, full forward and backward verified against PyTorch autograd to 1e-6.
☆19Mar 6, 2026Updated 6 months ago
Alternatives and similar repositories for FlashAttention-CuPy
Users that are interested in FlashAttention-CuPy are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- ☆12Oct 28, 2023Updated 2 years ago
- [CCS'24] Official Implementation of "Fisher Information guided Purification against Backdoor Attacks"☆15Oct 29, 2025Updated 10 months ago
- a minimal paged attention implementation☆20Jan 30, 2026Updated 7 months ago
- kaggle - RSNA STR Pulmonary Embolism Detection☆10Nov 22, 2020Updated 5 years ago
- Code for the paper "FedFisher: Leveraging Fisher Information for One-Shot Federated Learning" by Divyansh Jhunjhunwala, Shiqiang Wang, an…☆15Feb 10, 2024Updated 2 years ago
- GPUs on demand by Runpod - Special Offer Available • AdRun AI, ML, and HPC workloads on powerful cloud GPUs—without limits or wasted spend. Deploy GPUs in under a minute and pay by the second.
- The best ChatGPT that $100 can buy.☆58Sep 3, 2026Updated last week
- ☆14Nov 12, 2024Updated last year
- A GPT-2 inference engine written from scratch in CUDA and C++. Implements custom CUDA kernels for tiled matrix multiplication, LayerNorm,…☆44May 17, 2026Updated 3 months ago
- Single Cell Biology and Bioinformatics☆27Apr 25, 2025Updated last year
- [ICLR23] First deep learning-based surrogate model that jointly learns the evolution model and optimizes computational cost via remeshing☆42Jan 30, 2024Updated 2 years ago
- Python package for processing deep learning models☆16Dec 12, 2025Updated 9 months ago
- A visual representation of Dijkstra's Algorithm using Libgdx.☆16Dec 24, 2021Updated 4 years ago
- A neural network library written in jax☆14Feb 3, 2025Updated last year
- A block oriented training approach for inference time optimization.☆34Aug 19, 2024Updated 2 years ago
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- ☆10May 21, 2024Updated 2 years ago
- Official implementation of "NinA: Normalizing Flows in Action. Training VLA Models with Normalizing Flows"☆18Sep 22, 2025Updated 11 months ago
- Accompanying repo for the paper - High-speed Autonomous Racing using Trajectory-aided Deep Reinforcement Learning☆18Jan 17, 2024Updated 2 years ago
- Raw bindings to platform APIs for OCaml☆16Mar 18, 2024Updated 2 years ago
- ☆14Feb 9, 2026Updated 7 months ago
- this is the notes of cuda c++ programming guide, which hopes record my think in learning.☆15Sep 4, 2025Updated last year
- A test for RL application on f1tenth gym environment☆11Apr 10, 2023Updated 3 years ago
- 🚀 Collection of libraries used with fms-hf-tuning to accelerate fine-tuning and training of large models.☆14Sep 8, 2026Updated last week
- Improving AI Systems with Self-Defense Mechanisms☆25Feb 28, 2025Updated last year
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- clear single-file JAX implementations of common RL algorithms☆15Sep 5, 2021Updated 5 years ago
- Simply drag and drop your PDF files into Preve to get started. Ask Preve questions about your document. Get Summaries, key points, specif…☆11Apr 9, 2026Updated 5 months ago
- Adaptation of titans-pytorch to llama models on HF☆25Mar 6, 2025Updated last year
- ☆14May 25, 2023Updated 3 years ago
- ☆18Aug 14, 2023Updated 3 years ago
- A simple but instructive implementation of DP, TP, FSDP, FSDP+TP using pytorch distributed primitives☆24Apr 12, 2026Updated 5 months ago
- NURBSFit: Robust Fitting of NURBS Surfaces to Point Clouds☆16Mar 16, 2026Updated 5 months ago
- A unified framework for feed-forward neural networks☆15Nov 28, 2025Updated 9 months ago
- ☆15Jan 19, 2018Updated 8 years ago
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- A NodeJS application to upload, watch and stream live videos.☆12Jan 24, 2023Updated 3 years ago
- JAX implementations of various deep reinforcement learning algorithms.☆25Feb 2, 2025Updated last year
- Rust FTL + WebRTC live streaming software.☆13Mar 12, 2022Updated 4 years ago
- ☆16Feb 11, 2017Updated 9 years ago
- ☆19Sep 13, 2024Updated 2 years ago
- ☆27Jul 13, 2026Updated 2 months ago
- Microbenchmarking hyperparameter tuning for JAX functions.☆24Aug 24, 2026Updated 3 weeks ago