DeepSeek Native Sparse Attention pytorch implementation
β115Dec 17, 2025Updated 3 months ago
Alternatives and similar repositories for NSA-pytorch
Users that are interested in NSA-pytorch are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- π³ Efficient Triton implementations for "Native Sparse Attention: Hardware-Aligned and Natively Trainable Sparse Attention"β982Feb 5, 2026Updated 2 months ago
- Implementation of the sparse attention pattern proposed by the Deepseek team in their "Native Sparse Attention" paperβ800Aug 15, 2025Updated 7 months ago
- qwen-nsaβ87Oct 14, 2025Updated 5 months ago
- An efficient implementation of the NSA (Native Sparse Attention) kernelβ132Jun 24, 2025Updated 9 months ago
- LongAttn οΌSelecting Long-context Training Data via Token-level Attentionβ15Jul 16, 2025Updated 8 months ago
- 1-Click AI Models by DigitalOcean Gradient β’ AdDeploy popular AI models on DigitalOcean Gradient GPU virtual machines with just a single click and start building anything your business needs.
- High performance RMSNorm Implement by using SM Core Storage(Registers and Shared Memory)β30Jan 22, 2026Updated 2 months ago
- [NeurIPS 2024] Official implementation of paper "Perceiving Longer Sequences With Bi-Directional Cross-Attention Transformers"β20Mar 10, 2025Updated last year
- Code for paper: [ICLR2025 Oral] FlexPrefill: A Context-Aware Sparse Attention Mechanism for Efficient Long-Sequence Inferenceβ168Oct 13, 2025Updated 6 months ago
- Creating the DeepSeek V3 model from scratchβ27Mar 28, 2025Updated last year
- [ICLR 2025] DuoAttention: Efficient Long-Context LLM Inference with Retrieval and Streaming Headsβ535Feb 10, 2025Updated last year
- C++ library for finding Strongly Connected Components in parallel, based on paper: https://dl.acm.org/citation.cfm?id=2851161β12May 22, 2018Updated 7 years ago
- Sample Codes using NVSHMEM on Multi-GPUβ30Jan 22, 2023Updated 3 years ago
- Website for CSE 234, Winter 2025β13Mar 24, 2025Updated last year
- DeepSeek-V3.2-Exp DSA Warmup Lightning Indexer training operator based on tilelangβ44Nov 19, 2025Updated 4 months ago
- GPU virtual machines on DigitalOcean Gradient AI β’ AdGet to production fast with high-performance AMD and NVIDIA GPUs you can spin up in seconds. The definition of operational simplicity.
- Expert Specialization MoE Solution based on CUTLASSβ26Jan 19, 2026Updated 2 months ago
- This repo contains PyTorch implementation of cGAN, cWGAN, and cWGAN-gp for tabular data.β15Dec 21, 2023Updated 2 years ago
- Benchmark tests supporting the TiledCUDA library.β18Nov 19, 2024Updated last year
- β61Feb 5, 2026Updated 2 months ago
- [ICML2025] SpargeAttention: A training-free sparse attention that accelerates any model inference.β973Feb 25, 2026Updated last month
- β139May 29, 2025Updated 10 months ago
- TileGraph is an experimental DNN compiler that utilizes static code generation and kernel fusion techniques.β11Sep 18, 2024Updated last year
- Codebase for decoding compressed trust.β26May 7, 2024Updated last year
- CUDA SGEMM optimization noteβ15Oct 31, 2023Updated 2 years ago
- End-to-end encrypted cloud storage - Proton Drive β’ AdSpecial offer: 40% Off Yearly / 80% Off First Month. Protect your most important files, photos, and documents from prying eyes.
- Efficient LLM Inference over Long Sequencesβ393Jun 25, 2025Updated 9 months ago
- A collection of memory efficient attention operators implemented in the Triton language.β289Jun 5, 2024Updated last year
- analyse problems of AI with Math and Codeβ27Jul 28, 2025Updated 8 months ago
- A sparse attention kernel supporting mix sparse patternsβ495Jan 18, 2026Updated 2 months ago
- A PyTorch native library for training speculative decoding modelsβ76Updated this week
- MoBA: Mixture of Block Attention for Long-Context LLMsβ2,090Apr 3, 2025Updated last year
- β278Jun 6, 2025Updated 10 months ago
- [TPAMI2024] Learning to Holistically Detect Bridges from Large-Size VHR Remote Sensing Imageryβ15Mar 18, 2025Updated last year
- A lightweight reinforcement learning framework that integrates seamlessly into your codebase, empowering developers to focus on algorithmβ¦β103Aug 25, 2025Updated 7 months ago
- 1-Click AI Models by DigitalOcean Gradient β’ AdDeploy popular AI models on DigitalOcean Gradient GPU virtual machines with just a single click and start building anything your business needs.
- Nex Venus Communication Libraryβ73Nov 17, 2025Updated 4 months ago
- Wave: Python Domain-Specific Language for High Performance Machine Learningβ51Updated this week
- Official repository of Graph RAG-Tool Fusion and ToolLinkOS dataset.β23Feb 13, 2025Updated last year
- Design hardware-friendly model architectures and migrate existing LLMs with minimal performance lossβ453Updated this week
- β10Jun 19, 2023Updated 2 years ago
- β77Jun 23, 2025Updated 9 months ago
- Triton based sparse quantization attention kernel collectionβ43Aug 29, 2025Updated 7 months ago