Alternatives and similar repositories for fused_ops

Users that are interested in fused_ops are comparing it to the libraries listed below

• instance-wise-ordered-transformer / IOT
  ☆20Updated 4 years ago
• sIncerass / powernorm
  [ICML 2020] code for "PowerNorm: Rethinking Batch Normalization in Transformers" https://arxiv.org/abs/2003.07845
  ☆120Updated 4 years ago
• gonglinyuan / StackingBERT
  Source code for "Efficient Training of BERT by Progressively Stacking"
  ☆113Updated 6 years ago
• LAION-AI / Conditional-Pretraining-of-Large-Language-Models
  ☆37Updated 2 years ago
• NVIDIA / dllogger
  A logging tool for deep learning.
  ☆60Updated 6 months ago
• layer6ai-labs / T-Fixup
  Code for the ICML'20 paper "Improving Transformer Optimization Through Better Initialization"
  ☆89Updated 4 years ago
• mlpc-ucsd / BERT_Convolutions
  (ACL-IJCNLP 2021) Convolutions and Self-Attention: Re-interpreting Relative Positions in Pre-trained Language Models.
  ☆21Updated 3 years ago
• microsoft / Stochastic-Mixture-of-Experts
  This package implements THOR: Transformer with Stochastic Experts.
  ☆65Updated 3 years ago
• ahennequ / pytorch-custom-mma
  ☆29Updated 3 years ago
• CHARM-Tx / linear_mem_attention_pytorch
  Unofficially Implements https://arxiv.org/abs/2112.05682 to get Linear Memory Cost on Attention for PyTorch
  ☆12Updated 3 years ago
• HKUNLP / efficient-attention
  [EVA ICLR'23; LARA ICML'22] Efficient attention mechanisms via control variates, random features, and importance sampling
  ☆86Updated 2 years ago
• pytorch / torchdistx
  Torch Distributed Experimental
  ☆117Updated last year
• NVIDIA / LDDL
  Distributed preprocessing and data loading for language datasets
  ☆39Updated last year
• lucidrains / lie-transformer-pytorch
  Implementation of Lie Transformer, Equivariant Self-Attention, in Pytorch
  ☆96Updated 4 years ago
• Lightning-AI / forked-pdb
  Python pdb for multiple processes
  ☆59Updated 4 months ago
• shreyansh26 / Attention-Mask-Patterns
  Using FlexAttention to compute attention with different masking patterns
  ☆44Updated last year
• lucidrains / triangle-multiplicative-module
  Implementation of the Triangle Multiplicative module, used in Alphafold2 as an efficient way to mix rows or columns of a 2d feature map, …
  ☆36Updated 4 years ago
• MicPie / clasp
  CLASP - Contrastive Language-Aminoacid Sequence Pretraining
  ☆143Updated 4 years ago
• tbachlechner / ReZero-examples
  PyTorch Examples repo for "ReZero is All You Need: Fast Convergence at Large Depth"
  ☆62Updated last year
• kssteven418 / LTP
  [KDD'22] Learned Token Pruning for Transformers
  ☆100Updated 2 years ago
• lucidrains / rela-transformer
  Implementation of a Transformer using ReLA (Rectified Linear Attention) from https://arxiv.org/abs/2104.07012
  ☆49Updated 3 years ago
• lsj2408 / URPE
  [NeurIPS 2022] Your Transformer May Not be as Powerful as You Expect (official implementation)
  ☆33Updated 2 years ago
• ziplab / QLLM
  [ICLR 2024] This is the official PyTorch implementation of "QLLM: Accurate and Efficient Low-Bitwidth Quantization for Large Language Mod…
  ☆30Updated last year
• yaohungt / TransformerDissection
  [EMNLP'19] Summary for Transformer Understanding
  ☆53Updated 5 years ago
• lucidrains / learning-to-expire-pytorch
  An implementation of Transformer with Expire-Span, a circuit for learning which memories to retain
  ☆34Updated 4 years ago
• zomux / lanmt-ebm
  lanmt ebm
  ☆12Updated 5 years ago
• kyegomez / Blockwise-Parallel-Transformer
  32 times longer context window than vanilla Transformers and up to 4 times longer than memory efficient Transformers.
  ☆49Updated 2 years ago
• vvvm23 / sundae
  Unofficial PyTorch implementation of "Step-unrolled Denoising Autoencoders for Text Generation"
  ☆24Updated 2 years ago
• lucidrains / coco-lm-pytorch
  Implementation of COCO-LM, Correcting and Contrasting Text Sequences for Language Model Pretraining, in Pytorch
  ☆46Updated 4 years ago
• CyndxAI / QKNorm
  Code for the paper "Query-Key Normalization for Transformers"
  ☆48Updated 4 years ago