Alternatives and similar repositories for pytorch-ddp

Users that are interested in pytorch-ddp are comparing it to the libraries listed below

• lucidrains / tableformer-pytorch
  Implementation of TableFormer, Robust Transformer Modeling for Table-Text Encoding, in Pytorch
  ☆39Updated 3 years ago
• microsoft / AutoMoE
  AutoMoE: Neural Architecture Search for Efficient Sparsely Activated Transformers
  ☆47Updated 2 years ago
• knotgrass / attention
  several types of attention modules written in PyTorch for learning purposes
  ☆54Updated 8 months ago
• fkodom / soft-mixture-of-experts
  PyTorch implementation of Soft MoE by Google Brain in "From Sparse to Soft Mixtures of Experts" (https://arxiv.org/pdf/2308.00951.pdf)
  ☆73Updated last year
• schwartz-lab-NLP / papa
  Code for the PAPA paper
  ☆27Updated 2 years ago
• lucidrains / rela-transformer
  Implementation of a Transformer using ReLA (Rectified Linear Attention) from https://arxiv.org/abs/2104.07012
  ☆49Updated 3 years ago
• hoya012 / swa-tutorials-pytorch
  Stochastic Weight Averaging Tutorials using pytorch.
  ☆33Updated 4 years ago
• aniquetahir / JORA
  JORA: JAX Tensor-Parallel LoRA Library (ACL 2024)
  ☆34Updated last year
• vrvlive / knowlege-distillation
  PyTorch, PyTorch Lightning framework for trying knowledge distillation in image classification problems
  ☆32Updated 11 months ago
• pkuzengqi / Skyformer
  Skyformer: Remodel Self-Attention with Gaussian Kernel and Nystr\"om Method (NeurIPS 2021)
  ☆61Updated 3 years ago
• byeongjokim / VIPriors-Image-Classification-Challenge
  ☆17Updated 4 years ago
• bwconrad / soft-moe
  PyTorch implementation of "From Sparse to Soft Mixtures of Experts"
  ☆57Updated last year
• CyndxAI / QKNorm
  Code for the paper "Query-Key Normalization for Transformers"
  ☆41Updated 4 years ago
• fattorib / Flax-ResNets
  CIFAR10 ResNets implemented in JAX+Flax
  ☆12Updated 3 years ago
• LAION-AI / Conditional-Pretraining-of-Large-Language-Models
  ☆37Updated 2 years ago
• ahennequ / pytorch-custom-mma
  ☆29Updated 2 years ago
• rwightman / imagenet-12k
  ImageNet-12k subset of ImageNet-21k (fall11)
  ☆21Updated 2 years ago
• sarthmit / Mod_Arch
  ☆33Updated last year
• VITA-Group / Random-MoE-as-Dropout
  [ICLR 2023] "Sparse MoE as the New Dropout: Scaling Dense and Self-Slimmable Transformers" by Tianlong Chen*, Zhenyu Zhang*, Ajay Jaiswal…
  ☆52Updated 2 years ago
• shoaibahmed / llm_depth_pruning
  Official implementation of the paper: "A deeper look at depth pruning of LLMs"
  ☆15Updated 11 months ago
• amirzandieh / HyperAttention
  Triton Implementation of HyperAttention Algorithm
  ☆48Updated last year
• quentinf00 / article-memory-log
  Implémentation of the article **Deep Learning CUDA Memory Usage and Pytorch optimization tricks**
  ☆43Updated 5 years ago
• lucidrains / axial-positional-embedding
  Axial Positional Embedding for Pytorch
  ☆81Updated 4 months ago
• tascj / kaggle-feedback-prize-2021
  Solution of Kaggle competition: Feedback Prize - Evaluating Student Writing
  ☆16Updated 3 years ago
• Arenaa / Accelerated-Generation-Techniques
  This repository contains papers for a comprehensive survey on accelerated generation techniques in Large Language Models (LLMs).
  ☆11Updated last year
• lucidrains / infini-transformer-pytorch
  Implementation of Infini-Transformer in Pytorch
  ☆111Updated 5 months ago
• shreyansh26 / Attention-Mask-Patterns
  Using FlexAttention to compute attention with different masking patterns
  ☆44Updated 9 months ago
• Lightning-Universe / lightning-ColossalAI
  Large Scale Distributed Model Training strategy with Colossal AI and Lightning AI
  ☆57Updated last year
• juvi21 / CoPE-cuda
  Contextual Position Encoding but with some custom CUDA Kernels https://arxiv.org/abs/2405.18719
  ☆22Updated last year
• conceptofmind / t5-pytorch
  Implementation of Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer in PyTorch.
  ☆51Updated last year