Flash Attention from scratch, tiled CUDA forward kernel, online softmax with running max and correction factor, recomputation trick in backward, O(N) memory, full forward and backward verified against PyTorch autograd to 1e-6.
☆18Mar 6, 2026Updated 5 months ago
Alternatives and similar repositories for FlashAttention-CuPy
Users that are interested in FlashAttention-CuPy are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- [CCS'24] Official Implementation of "Fisher Information guided Purification against Backdoor Attacks"☆15Oct 29, 2025Updated 9 months ago
- kaggle - RSNA STR Pulmonary Embolism Detection☆10Nov 22, 2020Updated 5 years ago
- The best ChatGPT that $100 can buy.☆58Updated this week
- ☆26Sep 9, 2024Updated last year
- A minimalist quadcopter model and hover simulation built in MATLAB/Octave, based on Francesco Sabatino’s master thesis at KTH. This proje…☆12Aug 13, 2025Updated last year
- Managed Database hosting by DigitalOcean • AdPostgreSQL, MySQL, MongoDB, Kafka, Valkey, and OpenSearch available. Automatically scale up storage and focus on building your apps.
- A GPT-2 inference engine written from scratch in CUDA and C++. Implements custom CUDA kernels for tiled matrix multiplication, LayerNorm,…☆43May 17, 2026Updated 3 months ago
- Interactive version of the CuTe layout paper☆57Apr 14, 2026Updated 4 months ago
- [ICLR23] First deep learning-based surrogate model that jointly learns the evolution model and optimizes computational cost via remeshing☆42Jan 30, 2024Updated 2 years ago
- This repository contains PyTorch implementations of deep reinforcement learning algorithms and environments for Robotics and Controls. T…☆19Mar 20, 2022Updated 4 years ago
- A visual representation of Dijkstra's Algorithm using Libgdx.☆16Dec 24, 2021Updated 4 years ago
- A ROS package with nodes that implements drone detection using neural netowrk and depth maps.☆15Jan 15, 2025Updated last year
- A block oriented training approach for inference time optimization.☆34Aug 19, 2024Updated 2 years ago
- Multi-agent Reinforcement Learning game using Advantage Actor Critic (A2C) algorithm☆14Sep 26, 2023Updated 2 years ago
- This is the code repository of drone deep reinforcement learning☆12Mar 30, 2021Updated 5 years ago
- Wordpress hosting with auto-scaling - Free Trial Offer • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- Raw bindings to platform APIs for OCaml☆16Mar 18, 2024Updated 2 years ago
- A test for RL application on f1tenth gym environment☆11Apr 10, 2023Updated 3 years ago
- ☆11Nov 15, 2021Updated 4 years ago
- Mini RL Lab☆16Jun 17, 2024Updated 2 years ago
- ☆23Jan 14, 2025Updated last year
- Reverse engineered Twitter's API☆12Nov 28, 2023Updated 2 years ago
- 🚀 Collection of libraries used with fms-hf-tuning to accelerate fine-tuning and training of large models.☆14Jan 30, 2026Updated 6 months ago
- Improving AI Systems with Self-Defense Mechanisms☆24Feb 28, 2025Updated last year
- VS Code Extension for Kaggle☆23Dec 9, 2024Updated last year
- Wordpress hosting with auto-scaling - Free Trial Offer • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- ☆19Sep 16, 2024Updated last year
- Adaptation of titans-pytorch to llama models on HF☆25Mar 6, 2025Updated last year
- ☆14May 25, 2023Updated 3 years ago
- AI-queryable MCP server for unified real-time air, sea, and space tracking — 25+ tools, agent skills, guardrails, and a 3D globe☆19Apr 15, 2026Updated 4 months ago
- Simply drag and drop your PDF files into Preve to get started. Ask Preve questions about your document. Get Summaries, key points, specif…☆11Apr 9, 2026Updated 4 months ago
- ☆17Aug 14, 2023Updated 3 years ago
- A simple but instructive implementation of DP, TP, FSDP, FSDP+TP using pytorch distributed primitives☆23Apr 12, 2026Updated 4 months ago
- ☆15Jan 19, 2018Updated 8 years ago
- Collection of resources on plasticity loss in deep reinforcement learning☆23Nov 12, 2024Updated last year
- Serverless GPU API endpoints on Runpod - Get Bonus Credits • AdSkip the infrastructure headaches. Auto-scaling, pay-as-you-go, no-ops approach lets you focus on innovating your application.
- Rust FTL + WebRTC live streaming software.☆13Mar 12, 2022Updated 4 years ago
- CartPole-v0 via PPO with GAE, PyTorch☆22Feb 10, 2019Updated 7 years ago
- Website for the ICML 2021 tutorial on Random Matrix Theory and Machine Learning☆16Dec 8, 2021Updated 4 years ago
- ☆27Jul 13, 2026Updated last month
- Brax + Pufferlib + CARBS for gpu-accelerated robotics RL☆12Jun 12, 2025Updated last year
- Meta in-context learning for protein fitness prediction☆21Feb 7, 2025Updated last year
- Microbenchmarking hyperparameter tuning for JAX functions.☆24Updated this week