DeepSeek Native Sparse Attention pytorch implementation
☆120Dec 17, 2025Updated 9 months ago
Alternatives and similar repositories for NSA-pytorch
Users that are interested in NSA-pytorch are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- ☆48Dec 13, 2025Updated 9 months ago
- 🐳 Efficient Triton implementations for "Native Sparse Attention: Hardware-Aligned and Natively Trainable Sparse Attention"☆1,023Feb 5, 2026Updated 8 months ago
- PyTorch DeepSeek Sparse Attention (DSA) training & inference☆24Oct 14, 2025Updated 11 months ago
- Implementation of the sparse attention pattern proposed by the Deepseek team in their "Native Sparse Attention" paper☆816Aug 15, 2025Updated last year
- qwen-nsa☆87Oct 14, 2025Updated 11 months ago
- 1-Click AI Models by DigitalOcean Gradient • AdDeploy popular AI models on DigitalOcean Gradient GPU virtual machines with just a single click. Zero configuration with optimized deployments.
- An efficient implementation of the NSA (Native Sparse Attention) kernel☆135Jun 24, 2025Updated last year
- ☆104Feb 11, 2026Updated 8 months ago
- BigBang-Proton is a LLM pretrained on cross-scale, cross-structure, cross-discipline real-world scientific tasks to construct a scienti…☆21Nov 8, 2025Updated 11 months ago
- Code for paper: [ICLR2025 Oral] FlexPrefill: A Context-Aware Sparse Attention Mechanism for Efficient Long-Sequence Inference☆172Oct 13, 2025Updated 11 months ago
- High performance RMSNorm Implement by using SM Core Storage(Registers and Shared Memory)☆31Jan 22, 2026Updated 8 months ago
- Efficient triton implementation of Native Sparse Attention.☆283May 23, 2025Updated last year
- We introduce UltraLLaDA , a scaled variant of LLaDA-8B-Base that extends the context length up to 128K tokens with light-weight post-trai…☆15Oct 23, 2025Updated 11 months ago
- FlashTile is a CUDA Tile IR compiler that is compatible with NVIDIA's tileiras, targeting SM70 through SM121 NVIDIA GPUs.☆60Feb 6, 2026Updated 8 months ago
- ☆255Nov 19, 2025Updated 10 months ago
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- ☆37Aug 7, 2025Updated last year
- Some funny cute/cuteDSL code snippets☆35Mar 2, 2026Updated 7 months ago
- ☆155Mar 4, 2025Updated last year
- [ASP-DAC 2025] LightCL: Compact Continual Learning with Low Memory Footprint For Edge Device☆16Apr 2, 2025Updated last year
- A collection of memory efficient attention operators implemented in the Triton language.☆318Updated this week
- [AAAI 2026 Oral] SAMCL: Empowering SAM to Continually Learn from Dynamic Domains with Extreme Storage Efficiency☆21Mar 25, 2026Updated 6 months ago
- Visualize the Expert Parallelism Load Balancer☆17Mar 15, 2025Updated last year
- ☆46Oct 15, 2025Updated 11 months ago
- ☆17Jul 18, 2026Updated 2 months ago
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- [NeurIPS 2024] Official implementation of paper "Perceiving Longer Sequences With Bi-Directional Cross-Attention Transformers"☆22Mar 10, 2025Updated last year
- DeepSeek-V3.2-Exp DSA Warmup Lightning Indexer training operator based on tilelang☆52Nov 19, 2025Updated 10 months ago
- [ICLR 2025] DuoAttention: Efficient Long-Context LLM Inference with Retrieval and Streaming Heads☆541Feb 10, 2025Updated last year
- A Survey of Efficient Attention Methods: Hardware-efficient, Sparse, Compact, and Linear Attention☆311Dec 1, 2025Updated 10 months ago
- Design hardware-friendly model architectures and migrate existing LLMs with minimal performance loss☆512Updated this week
- 🚀 Efficient implementations for emerging model architectures☆5,842Updated this week
- The official repository for SkyLadder: Better and Faster Pretraining via Context Window Scheduling☆44Dec 29, 2025Updated 9 months ago
- pytorch implementation of DeepSeek Engram☆19Mar 24, 2026Updated 6 months ago
- Cute layout visualization☆45Jan 18, 2026Updated 8 months ago
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- TileFusion is an experimental C++ macro kernel template library that elevates the abstraction level in CUDA C for tile processing.☆119Aug 4, 2026Updated 2 months ago
- A tiny yet powerful LLM inference system tailored for researching purpose. vLLM-equivalent performance with only 2k lines of code (2% of …☆359Jun 10, 2025Updated last year
- ☆138May 29, 2025Updated last year
- [ICML2025] SpargeAttention: A training-free sparse attention that accelerates any model inference.☆1,260Feb 25, 2026Updated 7 months ago
- Efficient LLM Inference over Long Sequences☆392Jun 25, 2025Updated last year
- Distributed Compiler and Optimized Parallel Kernels☆1,561Sep 18, 2026Updated 3 weeks ago
- Accelerating MoE with IO and Tile-aware Optimizations☆778Aug 29, 2026Updated last month