PyTorch implementation of the Differential-Transformer architecture for sequence modeling, specifically tailored as a decoder-only model similar to large language models (LLMs). The architecture incorporates a novel Differential Attention mechanism, Multi-Head structure, RMSNorm, and SwiGLU.
☆86Oct 27, 2024Updated 2 years ago
Alternatives and similar repositories for Differential-Transformer-PyTorch
Users that are interested in Differential-Transformer-PyTorch are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- An open source community implementation of the model from "DIFFERENTIAL TRANSFORMER" paper by Microsoft.☆41Oct 26, 2026Updated 3 weeks ago
- ☆27Oct 12, 2024Updated 2 years ago
- Implementation of BitNet1.58b☆15Jul 9, 2024Updated 2 years ago
- Label Studio is a multi-type data labeling and annotation tool with standardized output format☆10Nov 17, 2021Updated 5 years ago
- ☆10Oct 27, 2023Updated 3 years ago
- GPU virtual machines on DigitalOcean Gradient AI • AdGet to production fast with high-performance AMD and NVIDIA GPUs you can spin up in seconds. The definition of operational simplicity.
- ☆10Sep 30, 2026Updated last month
- [CVPR 2025] Official PyTorch implementation of MaskSub "Masking meets Supervision: A Strong Learning Alliance"☆47Mar 25, 2025Updated last year
- Code repository for ‘Adaptive Differential Denoising for Respiratory Sounds Classification’☆24Dec 19, 2025Updated 11 months ago
- [ECCV2024] The code of "SHERL: Synthesizing High Accuracy and Efficient Memory for Resource-Limited Transfer Learning"☆10Oct 15, 2024Updated 2 years ago
- ☆10Dec 2, 2024Updated last year
- Official PyTorch Implementation of ZSLViT (CVPR'24)☆18Nov 7, 2026Updated last week
- Robotic Mobile Fulfillment System simulation to try different assignment and routing strategies in a robotic warehouse.☆10Nov 9, 2026Updated last week
- Code for the submitted 2021 DCASE Workshop paper: "Waveforms and Spectrograms: Enhancing Acoustic Scene Classification Using Multimodal F…☆16Aug 9, 2021Updated 5 years ago
- Pytorch implementation of paper "Contrastive Learning with Synthetic Positives"☆10Apr 24, 2025Updated last year
- 1-Click AI Models by DigitalOcean Gradient • AdDeploy popular AI models on DigitalOcean Gradient GPU virtual machines with just a single click. Zero configuration with optimized deployments.
- lncRNA-Py is a development package for applying machine learning and deep learning to the problem of lncRNA classification, i.e. predicti…☆12Jan 24, 2025Updated last year
- Pytorch implementation of DeMatch for CVPR'24 paper "DeMatch: Deep Decomposition of Motion Field for Two-View Correspondence Learning"☆13Jun 5, 2024Updated 2 years ago
- ☆12Feb 23, 2025Updated last year
- Griffin: Mixing Gated Linear Recurrences with Local Attention for Efficient Language Models☆15Oct 22, 2026Updated 3 weeks ago
- [ACCV 2024 (Oral, Best Application Paper)] Official Implementation of NT-VOT211: A Large-Scale Benchmark for Night-time Visual Object Tra…☆16Dec 30, 2025Updated 10 months ago
- ☆16Jul 24, 2024Updated 2 years ago
- [ICRA 2025] Fast Global Localization on Neural Radiance Field☆18Jun 30, 2025Updated last year
- Official PyTorch implementation of HCCNet: Efficient Semantic Matching with Hypercolumn Correlation (WACV '24 Oral, Best paper finalist (…☆11Apr 29, 2024Updated 2 years ago
- pytorch implements data enhancement and network regularization methods: cutmix, cutout, shakedrop, mixup, Label smoothing☆11Aug 19, 2021Updated 5 years ago
- Managed Database hosting by DigitalOcean • AdPostgreSQL, MySQL, MongoDB, Kafka, Valkey, and OpenSearch available. Automatically scale up storage and focus on building your apps.
- ☆10Mar 4, 2026Updated 8 months ago
- Code for the paper "No Train, all Gain: Self-Supervised Gradients Improve Deep Frozen Representations"☆11Oct 31, 2024Updated 2 years ago
- Chinese dense video captioning datasets☆10May 7, 2025Updated last year
- A PyTorch implementation of Determinantal Point Process Likelihoods for Sequential Recommendation☆12Dec 9, 2024Updated last year
- Genomic Tokenizer: Biology-driven Tokenizer for DNA sequences☆14Updated this week
- This repo contains VPR models that have been fine-tuned for indoor usage.☆16May 15, 2024Updated 2 years ago
- ☆13Oct 8, 2024Updated 2 years ago
- ☆12Jan 14, 2026Updated 10 months ago
- A benchmark suite of five genomics tasks for evaluating DNA foundation models on long-range dependencies.☆15Oct 6, 2026Updated last month
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- TRICARE for AI-triaged 3D pathology (Nature BME)☆12Aug 12, 2026Updated 3 months ago
- ☆16Dec 18, 2025Updated 11 months ago
- [NeurIPS25 Spotlight] Official Implementation for CBSA (Contract-and-Broadcast Self-Attention)☆37Apr 3, 2026Updated 7 months ago
- 为visinger SVS系统写的展示系统~本质仍然是个音乐播放器☆11Apr 18, 2023Updated 3 years ago
- [AAAI 26] Official repository for "D2VPR: A Parameter-efficient visual-foundation-model-based Visual Place Recognition Method via Knowled…☆14Updated this week
- Official implementation: "AudioSet-R: A Refined AudioSet with Multi-Stage LLM Label Reannotation"☆20Oct 9, 2025Updated last year
- [NeurIPS 2025] Rare Text Semantics Were Always There in Your Diffusion Transformer☆12Oct 7, 2025Updated last year
Are these results useful?