The-AI-Summer / pytorch-ddpLinks
code for the ddp tutorial
☆32Updated 3 years ago
Alternatives and similar repositories for pytorch-ddp
Users that are interested in pytorch-ddp are comparing it to the libraries listed below
Sorting:
- Implementation of TableFormer, Robust Transformer Modeling for Table-Text Encoding, in Pytorch☆39Updated 3 years ago
- Stochastic Weight Averaging Tutorials using pytorch.☆33Updated 4 years ago
- Code for the PAPA paper☆27Updated 2 years ago
- PyTorch implementation of Soft MoE by Google Brain in "From Sparse to Soft Mixtures of Experts" (https://arxiv.org/pdf/2308.00951.pdf)☆73Updated last year
- AutoMoE: Neural Architecture Search for Efficient Sparsely Activated Transformers☆46Updated 2 years ago
- [ICLR 2023] "Sparse MoE as the New Dropout: Scaling Dense and Self-Slimmable Transformers" by Tianlong Chen*, Zhenyu Zhang*, Ajay Jaiswal…☆51Updated 2 years ago
- Code for the paper "Query-Key Normalization for Transformers"☆41Updated 4 years ago
- ☆37Updated 2 years ago
- Skyformer: Remodel Self-Attention with Gaussian Kernel and Nystr\"om Method (NeurIPS 2021)☆61Updated 3 years ago
- PyTorch implementation of "From Sparse to Soft Mixtures of Experts"☆57Updated last year
- Implementation of a Transformer using ReLA (Rectified Linear Attention) from https://arxiv.org/abs/2104.07012☆50Updated 3 years ago
- JORA: JAX Tensor-Parallel LoRA Library (ACL 2024)☆33Updated last year
- A simple implementation of a deep linear Pytorch module☆21Updated 4 years ago
- An implementation of Transformer with Expire-Span, a circuit for learning which memories to retain☆34Updated 4 years ago
- Implementation of Infini-Transformer in Pytorch☆111Updated 5 months ago
- This repository contains papers for a comprehensive survey on accelerated generation techniques in Large Language Models (LLMs).☆11Updated last year
- The official repository for our paper "The Dual Form of Neural Networks Revisited: Connecting Test Time Predictions to Training Patterns …☆16Updated last year
- Contextual Position Encoding but with some custom CUDA Kernels https://arxiv.org/abs/2405.18719☆22Updated last year
- User-friendly implementation of the Mixture-of-Sparse-Attention (MoSA). MoSA selects distinct tokens for each head with expert choice rou…☆16Updated last month
- ☆20Updated 3 years ago
- ☆37Updated last year
- [NeurIPS 2023 spotlight] Official implementation of HGRN in our NeurIPS 2023 paper - Hierarchically Gated Recurrent Neural Network for Se…☆64Updated last year
- Reproducible code for Augmentation paper☆17Updated 6 years ago
- A pytorch realization of adafactor (https://arxiv.org/pdf/1804.04235.pdf )☆24Updated 5 years ago
- Triton Implementation of HyperAttention Algorithm☆48Updated last year
- Official implementation of the paper: "A deeper look at depth pruning of LLMs"☆15Updated 10 months ago
- PyTorch, PyTorch Lightning framework for trying knowledge distillation in image classification problems☆32Updated 10 months ago
- Implementation for ACProp ( Momentum centering and asynchronous update for adaptive gradient methdos, NeurIPS 2021)☆15Updated 3 years ago
- [EMNLP 2022] Official implementation of Transnormer in our EMNLP 2022 paper - The Devil in Linear Transformer☆60Updated last year
- ☆29Updated 2 years ago