PyTorch implementation of the Differential-Transformer architecture for sequence modeling, specifically tailored as a decoder-only model similar to large language models (LLMs). The architecture incorporates a novel Differential Attention mechanism, Multi-Head structure, RMSNorm, and SwiGLU.
☆86Oct 27, 2024Updated last year
Alternatives and similar repositories for Differential-Transformer-PyTorch
Users that are interested in Differential-Transformer-PyTorch are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- ☆21Updated this week
- My Implementation of Q-Sparse: All Large Language Models can be Fully Sparsely-Activated☆37Aug 14, 2024Updated 2 years ago
- A toolkit for researchers in the multimodal sound separation.☆16Oct 20, 2023Updated 2 years ago
- Exquisite video generation☆15Feb 18, 2024Updated 2 years ago
- noise reduction☆17Jul 3, 2024Updated 2 years ago
- Open source password manager - Proton Pass • AdSecurely store, share, and autofill your credentials with Proton Pass, the end-to-end encrypted password manager trusted by millions.
- 为visinger SVS系统写的展示系统～本质仍然是个音乐播放器☆11Apr 18, 2023Updated 3 years ago
- Automatic speech annotator processing speech with voice activaty detection, overlapping speech detection, speaker diarization and automat…☆33Jun 14, 2024Updated 2 years ago
- code for "Combating Noise: Semi-supervised Learning by Region Uncertainty Quantification"☆10Mar 19, 2022Updated 4 years ago
- Library for using deepseek api☆15Jan 28, 2025Updated last year
- [CVPR'24] Solving the Catastrophic Forgetting Problem in Generalized Category Discovery https://arxiv.org/pdf/2501.05272☆16Dec 24, 2024Updated last year
- ☆13Aug 8, 2026Updated 3 weeks ago
- ☆24Sep 25, 2024Updated last year
- This is the our implementation for the paper: Exploring Mixed Information Flow for Cross-domain Sequential Recommendations☆12Aug 17, 2020Updated 6 years ago
- Entity-Aware and Motion-Aware Transformers for Language-driven Action Localization(IJCAI-22)☆12Oct 11, 2022Updated 3 years ago
- Serverless GPU API endpoints on Runpod - Get Bonus Credits • AdSkip the infrastructure headaches. Auto-scaling, pay-as-you-go, no-ops approach lets you focus on innovating your application.
- Implementation of "Time Interval-enhanced Graph Neural Network for Shared-account Cross-domain Sequential Recommendation" (TNNLs 2022)☆12Mar 21, 2023Updated 3 years ago
- [NeurIPS 2024] Official repository of InLine attention☆62Dec 22, 2024Updated last year
- ☆11Dec 11, 2024Updated last year
- Med-DANet Series (ECCV 2022 & WACV 2024)☆13Jan 2, 2024Updated 2 years ago
- Sequence alignement methods with helpers for PyTorch.☆24Nov 30, 2022Updated 3 years ago
- C implementation of MATLAB built-in function of imresize.☆14Feb 12, 2018Updated 8 years ago
- The source code is for the paper: “Triple Sequence Learning for Cross-domain Recommendation” accepted in TOIS by Haokai Ma, Ruobing Xie, …☆18Mar 1, 2024Updated 2 years ago
- Exploration into the Scaling Value Iteration Networks paper, from Schmidhuber's group☆37Sep 23, 2024Updated last year
- Train a production grade GPT in less than 400 lines of code. Better than Karpathy's verison and GIGAGPT☆17Aug 28, 2026Updated last week
- Serverless GPU API endpoints on Runpod - Get Bonus Credits • AdSkip the infrastructure headaches. Auto-scaling, pay-as-you-go, no-ops approach lets you focus on innovating your application.
- NanoGPT (124M) in 5 minutes☆16Feb 14, 2025Updated last year
- Implementation of: Hydra Attention: Efficient Attention with Many Heads (https://arxiv.org/abs/2209.07484)☆14Jan 8, 2023Updated 3 years ago
- OmegaViT (ΩViT) is a cutting-edge vision transformer architecture that combines multi-query attention, rotary embeddings, state space mod…☆15Aug 28, 2026Updated last week
- This repo holds the official code and data for "Beyond Literal Descriptions: Understanding and Locating Open-World Objects Aligned with H…☆15May 21, 2024Updated 2 years ago
- [ICML 2025] Fourier Position Embedding: Enhancing Attention’s Periodic Extension for Length Generalization☆121Jun 2, 2025Updated last year
- Efficient Expert Pruning for Sparse Mixture-of-Experts Language Models: Enhancing Performance and Reducing Inference Costs☆25Nov 11, 2025Updated 9 months ago
- [ICLR 2022] "Audio Lottery: Speech Recognition Made Ultra-Lightweight, Noise-Robust, and Transferable", by Shaojin Ding, Tianlong Chen, Z…☆32Apr 8, 2022Updated 4 years ago
- Fork of Flame repo for training of some new stuff in development☆20Aug 27, 2026Updated last week
- MATLAB Image Processing Toolkit with 30+ features: transforms, filtering, edge detection, enhancement + a real-time user-friendly visuali…☆14Apr 25, 2026Updated 4 months ago
- Serverless GPU API endpoints on Runpod - Get Bonus Credits • AdSkip the infrastructure headaches. Auto-scaling, pay-as-you-go, no-ops approach lets you focus on innovating your application.
- This repository contains prompts & best practices to annotate audio clips with a very high degree of details using Audio-Language-Models☆35Oct 13, 2024Updated last year
- WWW 2023 "Mutual Wasserstein Discrepancy Minimization for Sequential Recommendation"☆16Jun 20, 2023Updated 3 years ago
- MLLMSeg: Unlocking the Potential of MLLMs in Referring Expression Segmentation via a Light-weight Mask Decoder☆57Jun 12, 2026Updated 2 months ago
- Official respository for ECCV24 paper "Diffusion-Guided Weakly Supervised Semantic Segmentation"☆21Dec 17, 2024Updated last year
- ☆16Jul 17, 2025Updated last year
- ☆18May 7, 2025Updated last year
- MediaPipeを用いたハンドジェスチャーによる簡単なマウス操作を行うプログラムです。☆12Mar 17, 2021Updated 5 years ago