DeepSeek Native Sparse Attention pytorch implementation
☆121Dec 17, 2025Updated 10 months ago
Alternatives and similar repositories for NSA-pytorch
Users that are interested in NSA-pytorch are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- qwen-nsa☆87Oct 14, 2025Updated last year
- 🐳 Efficient Triton implementations for "Native Sparse Attention: Hardware-Aligned and Natively Trainable Sparse Attention"☆1,026Feb 5, 2026Updated 8 months ago
- Implementation of the sparse attention pattern proposed by the Deepseek team in their "Native Sparse Attention" paper☆816Aug 15, 2025Updated last year
- ☆48Dec 13, 2025Updated 10 months ago
- PyTorch DeepSeek Sparse Attention (DSA) training & inference☆24Oct 14, 2025Updated last year
- Serverless GPU API endpoints on Runpod - Get Bonus Credits • AdSkip the infrastructure headaches. Auto-scaling, pay-as-you-go, no-ops approach lets you focus on innovating your application.
- A lightweight CuTe-based CUDA kernel for DuoAttention, optimized for large language model inference.☆12Dec 25, 2025Updated 10 months ago
- ☆104Feb 11, 2026Updated 8 months ago
- An efficient implementation of the NSA (Native Sparse Attention) kernel☆135Jun 24, 2025Updated last year
- BigBang-Proton is a LLM pretrained on cross-scale, cross-structure, cross-discipline real-world scientific tasks to construct a scienti…☆21Nov 8, 2025Updated 11 months ago
- Creating the DeepSeek V3 model from scratch☆30Oct 8, 2026Updated 3 weeks ago
- ☆156Mar 4, 2025Updated last year
- Code for paper: [ICLR2025 Oral] FlexPrefill: A Context-Aware Sparse Attention Mechanism for Efficient Long-Sequence Inference☆172Oct 13, 2025Updated last year
- We introduce UltraLLaDA , a scaled variant of LLaDA-8B-Base that extends the context length up to 128K tokens with light-weight post-trai…☆15Oct 23, 2025Updated last year
- Efficient triton implementation of Native Sparse Attention.☆285May 23, 2025Updated last year
- Managed Kubernetes at scale on DigitalOcean • AdDigitalOcean Kubernetes includes the control plane, bandwidth allowance, container registry, automatic updates, and more for free.
- High performance RMSNorm Implement by using SM Core Storage(Registers and Shared Memory)☆31Jan 22, 2026Updated 9 months ago
- FlashTile is a CUDA Tile IR compiler that is compatible with NVIDIA's tileiras, targeting SM70 through SM121 NVIDIA GPUs.☆62Feb 6, 2026Updated 8 months ago
- analyse problems of AI with Math and Code☆31Jul 28, 2025Updated last year
- Research on DeepSeek Sparse Attention☆40Oct 8, 2025Updated last year
- ☆255Nov 19, 2025Updated 11 months ago
- A lightweight SGLang implementation built from scratch.☆12Jul 29, 2025Updated last year
- [ICLR 2025] DuoAttention: Efficient Long-Context LLM Inference with Retrieval and Streaming Heads☆544Feb 10, 2025Updated last year
- ☆37Aug 7, 2025Updated last year
- [ASP-DAC 2025] LightCL: Compact Continual Learning with Low Memory Footprint For Edge Device☆16Apr 2, 2025Updated last year
- Virtual machines for every use case on DigitalOcean • AdGet dependable uptime with 99.99% SLA, simple security tools, and predictable monthly pricing with DigitalOcean's virtual machines, called Droplets.
- Enable Next-sentence Prediction for Large Language Models with Faster Speed, Higher Accuracy and Longer Context☆44Aug 16, 2024Updated 2 years ago
- Some funny cute/cuteDSL code snippets☆35Mar 2, 2026Updated 7 months ago
- ☆43Updated this week
- [AAAI 2026 Oral] SAMCL: Empowering SAM to Continually Learn from Dynamic Domains with Extreme Storage Efficiency☆21Mar 25, 2026Updated 7 months ago
- ☆20Aug 9, 2026Updated 2 months ago
- Nano vLLM☆15Jun 1, 2026Updated 4 months ago
- ☆18Jul 18, 2026Updated 3 months ago
- ☆46Oct 15, 2025Updated last year
- A Survey of Efficient Attention Methods: Hardware-efficient, Sparse, Compact, and Linear Attention☆314Dec 1, 2025Updated 11 months ago
- GPUs on demand by Runpod - Special Offer Available • AdRun AI, ML, and HPC workloads on powerful cloud GPUs—without limits or wasted spend. Deploy GPUs in under a minute and pay by the second.
- MoBA: Mixture of Block Attention for Long-Context LLMs☆2,197Apr 3, 2025Updated last year
- A collection of memory efficient attention operators implemented in the Triton language.☆327Updated this week
- 🚀 Efficient implementations for emerging model architectures☆6,008Updated this week
- DeepSeek-V3.2-Exp DSA Warmup Lightning Indexer training operator based on tilelang☆52Nov 19, 2025Updated 11 months ago
- The official repository for SkyLadder: Better and Faster Pretraining via Context Window Scheduling☆44Dec 29, 2025Updated 10 months ago
- ☆10Nov 18, 2024Updated last year
- Design hardware-friendly model architectures and migrate existing LLMs with minimal performance loss☆515Oct 23, 2026Updated last week