Flash Attention from scratch, tiled CUDA forward kernel, online softmax with running max and correction factor, recomputation trick in backward, O(N) memory, full forward and backward verified against PyTorch autograd to 1e-6.
☆18Mar 6, 2026Updated 4 months ago
Alternatives and similar repositories for FlashAttention-CuPy
Users that are interested in FlashAttention-CuPy are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- kaggle - RSNA STR Pulmonary Embolism Detection☆10Nov 22, 2020Updated 5 years ago
- Text Normalization utilities for normalizing text for TTS☆26Mar 4, 2026Updated 4 months ago
- Integrates Imbue's Cost Aware pareto-Region Bayesian Search (CARBS) with Weights and Biases (WanDB)☆12Mar 17, 2025Updated last year
- ☆25Sep 9, 2024Updated last year
- A GPT-2 inference engine written from scratch in CUDA and C++. Implements custom CUDA kernels for tiled matrix multiplication, LayerNorm,…☆40May 17, 2026Updated last month
- Managed Kubernetes at scale on DigitalOcean • AdDigitalOcean Kubernetes includes the control plane, bandwidth allowance, container registry, automatic updates, and more for free.
- 3rd Place solution☆12Nov 20, 2024Updated last year
- Interactive version of the CuTe layout paper☆57Apr 14, 2026Updated 3 months ago
- [ 👾 ] ➡️ 💾 ➡️ { 🎮🕹️ } Extra Stable-Baselines3 buffer classes. Reducing RL memory usage drastically with minimal overhead.☆24Jun 9, 2026Updated last month
- ☆39Jun 9, 2026Updated last month
- The official repository for paper: Agentic Robot: A Brain-Inspired Framework for Vision-Language-Action Models in Embodied Agents☆52Jun 22, 2025Updated last year
- A neural network library written in jax☆14Feb 3, 2025Updated last year
- Code for the paper "Stable Gradients for Stable Learning at Scale in Deep Reinforcement Learning". Great performance in many environments…☆39Oct 24, 2025Updated 8 months ago
- Multi-agent Reinforcement Learning game using Advantage Actor Critic (A2C) algorithm☆14Sep 26, 2023Updated 2 years ago
- A block oriented training approach for inference time optimization.☆34Aug 19, 2024Updated last year
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- Improving AI Systems with Self-Defense Mechanisms☆23Feb 28, 2025Updated last year
- An agent for CUDA compute-communication kernel co-design☆35May 7, 2026Updated 2 months ago
- Accompanying repo for the paper - High-speed Autonomous Racing using Trajectory-aided Deep Reinforcement Learning☆18Jan 17, 2024Updated 2 years ago
- Proposed plumbing commands for cargo☆26Jul 9, 2026Updated last week
- AI-queryable MCP server for unified real-time air, sea, and space tracking — 25+ tools, agent skills, guardrails, and a 3D globe☆15Apr 15, 2026Updated 3 months ago
- ☆11Nov 15, 2021Updated 4 years ago
- [AAAI 2025 Oral] Official code for "RAT: Adversarial Attacks on Deep Reinforcement Agents for Targeted Behaviors"☆34Feb 15, 2025Updated last year
- 🚀 Collection of libraries used with fms-hf-tuning to accelerate fine-tuning and training of large models.☆14Jan 30, 2026Updated 5 months ago
- clear single-file JAX implementations of common RL algorithms☆15Sep 5, 2021Updated 4 years ago
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- Simply drag and drop your PDF files into Preve to get started. Ask Preve questions about your document. Get Summaries, key points, specif…☆11Apr 9, 2026Updated 3 months ago
- Adaptation of titans-pytorch to llama models on HF☆24Mar 6, 2025Updated last year
- Library containing safer alternatives/wrappers for insecure C APIs.☆24Apr 28, 2026Updated 2 months ago
- AgentFence is an open-source platform for automatically testing AI agent security. It identifies vulnerabilities such as prompt injection…☆59Mar 6, 2025Updated last year
- ☆18Aug 14, 2023Updated 2 years ago
- A NodeJS application to upload, watch and stream live videos.☆12Jan 24, 2023Updated 3 years ago
- My attempt to improve the speed of the newton schulz algorithm, starting from the dion implementation.☆38Apr 30, 2026Updated 2 months ago
- Rust FTL + WebRTC live streaming software.☆13Mar 12, 2022Updated 4 years ago
- ☆25May 23, 2025Updated last year
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- A simple but instructive implementation of DP, TP, FSDP, FSDP+TP using pytorch distributed primitives☆19Apr 12, 2026Updated 3 months ago
- IAttorney is an intelligent legal assistant built using Flask, LangChain, FAISS, OpenAI, and a RAG (Retrieval-Augmented Generation) pipel…☆20Aug 6, 2025Updated 11 months ago
- ☆15Jan 19, 2018Updated 8 years ago
- Collection of resources on plasticity loss in deep reinforcement learning☆23Nov 12, 2024Updated last year
- JAX implementations of various deep reinforcement learning algorithms.☆25Feb 2, 2025Updated last year
- Meta-harness optimization loop wired onto Islo sandboxes. POC: 0/5→5/5 in four proposer steps. Built on islo.dev.☆25May 5, 2026Updated 2 months ago
- Brax + Pufferlib + CARBS for gpu-accelerated robotics RL☆12Jun 12, 2025Updated last year