several types of attention modules written in PyTorch for learning purposes
☆54Jan 2, 2026Updated 4 months ago
Alternatives and similar repositories for attention
Users that are interested in attention are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- (Unofficial) PyTorch implementation of grouped-query attention (GQA) from "GQA: Training Generalized Multi-Query Transformer Models from …☆194May 9, 2024Updated 2 years ago
- Script for processing OpenAI's PRM800K process supervision dataset into an Alpaca-style instruction-response format☆27Jul 12, 2023Updated 2 years ago
- Implementation of a Hierarchical Mamba as described in the paper: "Hierarchical State Space Models for Continuous Sequence-to-Sequence Mo…☆15Nov 11, 2024Updated last year
- ☆16Nov 4, 2023Updated 2 years ago
- This is a simple torch implementation of the high performance Multi-Query Attention☆16Aug 23, 2023Updated 2 years ago
- GPU virtual machines on DigitalOcean Gradient AI • AdGet to production fast with high-performance AMD and NVIDIA GPUs you can spin up in seconds. The definition of operational simplicity.
- Implementation of Variance Reduction Techniques in Julia☆11Sep 6, 2016Updated 9 years ago
- The official repo of Qwen2-Audio chat & pretrained large audio language model proposed by Alibaba Cloud.☆40Sep 8, 2024Updated last year
- Efficient implementation of Generative Stochastic Networks☆12Nov 28, 2013Updated 12 years ago
- LLM - Detect AI Generated Text || Identify which essay was written by a large language model☆17Jan 17, 2024Updated 2 years ago
- sigma-MoE layer☆21Jan 5, 2024Updated 2 years ago
- InternLM-7B微调, SFT/LoRA, instruction finetune☆13May 17, 2024Updated 2 years ago
- Code for the article series on building a Python compiler and interpreter☆11Feb 13, 2025Updated last year
- [CVPR2026] BinaryAttention: One-Bit QK-Attention for Vision and Diffusion Transformers☆33Mar 17, 2026Updated 2 months ago
- ☆11Aug 4, 2022Updated 3 years ago
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- ☆10Feb 19, 2019Updated 7 years ago
- The tool facilitates debugging convergence issues and testing new algorithms and recipes for training LLMs using Nvidia libraries such as…☆19Sep 17, 2025Updated 8 months ago
- The open source implementation of the multi grouped query attention by the paper "GQA: Training Generalized Multi-Query Transformer Model…☆16Dec 11, 2023Updated 2 years ago
- GC-Net: Global Attention Module and Cascade Fusion Network for Steel Surface Defect Detection☆10Dec 2, 2024Updated last year
- Official PyTorch code for HILA☆28Nov 1, 2022Updated 3 years ago
- Graph transport network (GTN), as proposed in "Scalable Optimal Transport in High Dimensions for Graph Distances, Embedding Alignment, an…☆15Apr 26, 2023Updated 3 years ago
- [AAAI'25] "Decomposed Spatio-Temporal Mamba for Long-Term Traffic Prediction"☆21May 30, 2025Updated 11 months ago
- Official Implementation for the paper "VisCodex: Unified Multimodal Code Generation via Merging Vision and Coding Models"☆22Aug 14, 2025Updated 9 months ago
- A clean implementation based on Expert Iterations for any game, inspired by alpha-zero-general☆46Dec 27, 2022Updated 3 years ago
- GPU virtual machines on DigitalOcean Gradient AI • AdGet to production fast with high-performance AMD and NVIDIA GPUs you can spin up in seconds. The definition of operational simplicity.
- [ICLR 2024] This is the official PyTorch implementation of "QLLM: Accurate and Efficient Low-Bitwidth Quantization for Large Language Mod…☆40Mar 11, 2024Updated 2 years ago
- Source Code for Online Collective Matrix Factorization Hashing. Reference: Di Wang, Quan Wang, Yaqiang An, Xinbo Gao, and Yumin Tian. 202…☆11Oct 20, 2020Updated 5 years ago
- Code for the paper "OTRE: Where Optimal Transport Guided Unpaired Image-to-Image Translation Meets Regularization by Enhancing"☆11Aug 2, 2025Updated 9 months ago
- TFSCL☆21Jul 25, 2024Updated last year
- Joint optic disc and optic cup segmentation based on boundary prior and adversarial learning☆15Jan 18, 2023Updated 3 years ago
- ☆16Jun 14, 2024Updated last year
- Code repository for the 2023 MICCAI Paper "TabAttention: Learning Attention Conditionally on Tabular Data"☆18Oct 30, 2023Updated 2 years ago
- React全家桶+AntD 共享单车后台管理系统开发☆14Aug 1, 2018Updated 7 years ago
- SHUOSC RISC-V implementation☆11May 12, 2024Updated 2 years ago
- Virtual machines for every use case on DigitalOcean • AdGet dependable uptime with 99.99% SLA, simple security tools, and predictable monthly pricing with DigitalOcean's virtual machines, called Droplets.
- A Sobol sequence generator for Scala and Javascript☆15Feb 15, 2020Updated 6 years ago
- Boundary-enhanced dual-stream network for semantic segmentation of high-resolution remote sensing images, 2024☆14Jun 26, 2024Updated last year
- 在module level分析模型的性能☆12Jun 21, 2025Updated 10 months ago
- ☆19Sep 15, 2022Updated 3 years ago
- Semi-Supervised Change Detection with Feature-Prediction Alignment☆11Sep 14, 2023Updated 2 years ago
- Spatial Spectral Machine Learning☆14Oct 15, 2025Updated 7 months ago
- "B2CNet: A Progressive Change Boundary-to-Center Refinement Network for Multitemporal Remote Sensing Images Change Detection", JSTARS 202…☆19Jun 27, 2024Updated last year