The-AI-Summer / pytorch-ddp
code for the ddp tutorial
☆32Updated 2 years ago
Related projects ⓘ
Alternatives and complementary repositories for pytorch-ddp
- Code for the PAPA paper☆27Updated 2 years ago
- Implementation of TableFormer, Robust Transformer Modeling for Table-Text Encoding, in Pytorch☆36Updated 2 years ago
- PyTorch, PyTorch Lightning framework for trying knowledge distillation in image classification problems☆32Updated 3 months ago
- Code for the paper "Query-Key Normalization for Transformers"☆35Updated 3 years ago
- Stochastic Weight Averaging Tutorials using pytorch.☆33Updated 4 years ago
- PyTorch implementation of "From Sparse to Soft Mixtures of Experts"☆47Updated last year
- PyTorch implementation of Soft MoE by Google Brain in "From Sparse to Soft Mixtures of Experts" (https://arxiv.org/pdf/2308.00951.pdf)☆66Updated last year
- Contextual Position Encoding but with some custom CUDA Kernels https://arxiv.org/abs/2405.18719☆20Updated 5 months ago
- ☆36Updated last year
- ☆29Updated 2 years ago
- The official implementation of You Only Compress Once: Towards Effective and Elastic BERT Compression via Exploit-Explore Stochastic Natu…☆48Updated 3 years ago
- Official code for Group-Transformer (Scale down Transformer by Grouping Features for a Lightweight Character-level Language Model, COLING…☆25Updated 3 years ago
- Implementation of COCO-LM, Correcting and Contrasting Text Sequences for Language Model Pretraining, in Pytorch☆45Updated 3 years ago
- [ICLR 2023] "Sparse MoE as the New Dropout: Scaling Dense and Self-Slimmable Transformers" by Tianlong Chen*, Zhenyu Zhang*, Ajay Jaiswal…☆44Updated last year
- An adaptive training algorithm for residual network☆14Updated 4 years ago
- Implementation of a Transformer using ReLA (Rectified Linear Attention) from https://arxiv.org/abs/2104.07012☆49Updated 2 years ago
- [ACL 2023] Code for paper “Tailoring Instructions to Student’s Learning Levels Boosts Knowledge Distillation”(https://arxiv.org/abs/2305.…☆37Updated last year
- An implementation of Transformer with Expire-Span, a circuit for learning which memories to retain☆33Updated 4 years ago
- PyTorch implementation of moe, which stands for mixture of experts☆32Updated 3 years ago
- Residual Prompt Tuning: a method for faster and better prompt tuning.☆50Updated last year
- ☆29Updated 2 years ago
- Skyformer: Remodel Self-Attention with Gaussian Kernel and Nystr\"om Method (NeurIPS 2021)☆59Updated 2 years ago
- Axial Positional Embedding for Pytorch☆60Updated 3 years ago
- A simple implementation of a deep linear Pytorch module☆18Updated 4 years ago
- ☆21Updated last year
- ☆45Updated 4 months ago
- Implementation of Memformer, a Memory-augmented Transformer, in Pytorch☆106Updated 4 years ago
- The official repository for our paper "The Dual Form of Neural Networks Revisited: Connecting Test Time Predictions to Training Patterns …☆16Updated last year
- ☆15Updated last year