Flash Attention from scratch, tiled CUDA forward kernel, online softmax with running max and correction factor, recomputation trick in backward, O(N) memory, full forward and backward verified against PyTorch autograd to 1e-6.
☆19Mar 6, 2026Updated 7 months ago
Alternatives and similar repositories for FlashAttention-CuPy
Users that are interested in FlashAttention-CuPy are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- So, I trained a MoE based a 124M (8x12M) architecture I coded from ground up to build a small instruct model, going through the below-men…☆14Jul 18, 2025Updated last year
- Comparative study and experimentation on standard vs mHC vs attention residual (full and block)☆13Mar 17, 2026Updated 7 months ago
- ☆11Apr 20, 2020Updated 6 years ago
- The best ChatGPT that $100 can buy.☆60Updated this week
- 【GRSL 2021】ASEA for Change Detection☆11Sep 1, 2026Updated last month
- Wordpress hosting with auto-scaling - Free Trial Offer • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- Change Detection network using Correlation Layer☆12Jan 19, 2020Updated 6 years ago
- An unofficial implementation of MEME (Efficient Memory-based Exploration agent) from DeepMind☆11Apr 13, 2024Updated 2 years ago
- This project implements knowledge distillation from DINOv2 (Vision Transformer) to convolutional networks, enabling efficient visual repr…☆12Jul 30, 2025Updated last year
- CuPy Benchmark☆13Apr 15, 2019Updated 7 years ago
- ☆16Nov 12, 2024Updated last year
- Interactive version of the CuTe layout paper☆56Apr 14, 2026Updated 6 months ago
- In-depth tutorials and examples on LLM training and inference infrastructure, such as, Pytorch, Fairscale, Nvidia AI Modules (cuDNN, tens…☆22May 19, 2025Updated last year
- Code for paper "Understanding Generalization, Robustness, and Interpretability in Low-Capacity Neural Networks"☆13Jul 24, 2025Updated last year
- microCMS Ruby SDK.☆10Oct 15, 2026Updated 2 weeks ago
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- A minimalist quadcopter model and hover simulation built in MATLAB/Octave, based on Francesco Sabatino’s master thesis at KTH. This proje…☆13Aug 13, 2025Updated last year
- A visual representation of Dijkstra's Algorithm using Libgdx.☆16Dec 24, 2021Updated 4 years ago
- A Minecraft mod that resets your world when all online players die.☆11May 31, 2026Updated 4 months ago
- A SOTA vision model built on top of llama3 8B.☆14May 28, 2024Updated 2 years ago
- Soft Actor-Critic implementation with SOTA model-free extension (REDQ) and SOTA model-based extension (MBPO).☆16Feb 21, 2021Updated 5 years ago
- Kaggle Notebooks☆12May 10, 2026Updated 5 months ago
- Implementation and explorations into Blackbox Gradient Sensing (BGS), an evolutionary strategies approach proposed in a Google Deepmind p…☆20Apr 17, 2026Updated 6 months ago
- A neural network library written in jax☆14Feb 3, 2025Updated last year
- ☆11Sep 26, 2023Updated 3 years ago
- Simple, predictable pricing with DigitalOcean hosting • AdAlways know what you'll pay with monthly caps and flat pricing. Enterprise-grade infrastructure trusted by 600k+ customers.
- This repository contains PyTorch implementations of deep reinforcement learning algorithms and environments for Robotics and Controls. T…☆19Mar 20, 2022Updated 4 years ago
- Mini RL Lab☆16Jun 17, 2024Updated 2 years ago
- Multi-agent Reinforcement Learning game using Advantage Actor Critic (A2C) algorithm☆14Sep 26, 2023Updated 3 years ago
- Official implementation of "NinA: Normalizing Flows in Action. Training VLA Models with Normalizing Flows"☆19Sep 22, 2025Updated last year
- Accompanying repo for the paper - High-speed Autonomous Racing using Trajectory-aided Deep Reinforcement Learning☆18Jan 17, 2024Updated 2 years ago
- A test for RL application on f1tenth gym environment☆11Apr 10, 2023Updated 3 years ago
- Reproducibility study of the paper: "InPars Toolkit: A Unified and Reproducible Synthetic Data Generation Pipeline for Neural Information…☆12Feb 11, 2025Updated last year
- A tool to simplify SVG files to use more basic features☆14Sep 16, 2026Updated last month
- implementation of a context graph?? we'll see☆11Dec 30, 2025Updated 9 months ago
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- ☆11Nov 15, 2021Updated 4 years ago
- clear single-file JAX implementations of common RL algorithms☆15Sep 5, 2021Updated 5 years ago
- ☆30Oct 15, 2026Updated 2 weeks ago
- ☆13May 25, 2023Updated 3 years ago
- Deep Networks Grok All the Time and Here is Why☆42Apr 20, 2026Updated 6 months ago
- Folia fork that runs one Minecraft world across several servers (Storia Cluster), with seamless moves between them. Also RAM worlds, fast…☆23Oct 6, 2026Updated 3 weeks ago
- Reverse engineered Twitter's API☆12Nov 28, 2023Updated 2 years ago