Alternatives and similar repositories for fused_ops

Users that are interested in fused_ops are comparing it to the libraries listed below

• instance-wise-ordered-transformer / IOT
  ☆20Updated 4 years ago
• CHARM-Tx / linear_mem_attention_pytorch
  Unofficially Implements https://arxiv.org/abs/2112.05682 to get Linear Memory Cost on Attention for PyTorch
  ☆12Updated 3 years ago
• NVIDIA / dllogger
  A logging tool for deep learning.
  ☆63Updated 8 months ago
• sIncerass / powernorm
  [ICML 2020] code for "PowerNorm: Rethinking Batch Normalization in Transformers" https://arxiv.org/abs/2003.07845
  ☆120Updated 4 years ago
• NVIDIA / LDDL
  Distributed preprocessing and data loading for language datasets
  ☆39Updated last year
• pytorch / torchdistx
  Torch Distributed Experimental
  ☆117Updated last year
• LAION-AI / Conditional-Pretraining-of-Large-Language-Models
  ☆37Updated 2 years ago
• mlpc-ucsd / BERT_Convolutions
  (ACL-IJCNLP 2021) Convolutions and Self-Attention: Re-interpreting Relative Positions in Pre-trained Language Models.
  ☆21Updated 3 years ago
• Gleghorn-Lab / Mixture-of-Experts-Sentence-Similarity
  ☆16Updated 9 months ago
• Lightning-AI / forked-pdb
  Python pdb for multiple processes
  ☆70Updated 6 months ago
• microsoft / Stochastic-Mixture-of-Experts
  This package implements THOR: Transformer with Stochastic Experts.
  ☆65Updated 4 years ago
• cheneydon / efficient-bert
  This repository contains the code for the paper in Findings of EMNLP 2021: "EfficientBERT: Progressively Searching Multilayer Perceptron …
  ☆33Updated 2 years ago
• lucidrains / triangle-multiplicative-module
  Implementation of the Triangle Multiplicative module, used in Alphafold2 as an efficient way to mix rows or columns of a 2d feature map, …
  ☆39Updated 4 years ago
• asappresearch / flop
  Pytorch library for factorized L0-based pruning.
  ☆45Updated 2 years ago
• XuezheMax / fairseq-apollo
  FairSeq repo with Apollo optimizer
  ☆114Updated 2 years ago
• juvi21 / CoPE-cuda
  Contextual Position Encoding but with some custom CUDA Kernels https://arxiv.org/abs/2405.18719
  ☆22Updated last year
• CyndxAI / QKNorm
  Code for the paper "Query-Key Normalization for Transformers"
  ☆49Updated 4 years ago
• huggingface / block_movement_pruning
  Block Sparse movement pruning
  ☆81Updated 5 years ago
• MAC-AutoML / YOCO-BERT
  The official implementation of You Only Compress Once: Towards Effective and Elastic BERT Compression via Exploit-Explore Stochastic Natu…
  ☆48Updated 4 years ago
• kssteven418 / LTP
  [KDD'22] Learned Token Pruning for Transformers
  ☆102Updated 2 years ago
• gonglinyuan / StackingBERT
  Source code for "Efficient Training of BERT by Progressively Stacking"
  ☆113Updated 6 years ago
• Lightning-Universe / lightning-ColossalAI
  Large Scale Distributed Model Training strategy with Colossal AI and Lightning AI
  ☆56Updated 2 years ago
• layer6ai-labs / T-Fixup
  Code for the ICML'20 paper "Improving Transformer Optimization Through Better Initialization"
  ☆89Updated 4 years ago
• tbachlechner / ReZero-examples
  PyTorch Examples repo for "ReZero is All You Need: Fast Convergence at Large Depth"
  ☆62Updated last year
• mit-han-lab / neurips-micronet
  [JMLR'20] NeurIPS 2019 MicroNet Challenge Efficient Language Modeling, Champion
  ☆41Updated 4 years ago
• lucidrains / product-key-memory
  Standalone Product Key Memory module in Pytorch - for augmenting Transformer models
  ☆87Updated last month
• ahennequ / pytorch-custom-mma
  ☆29Updated 3 years ago
• shreyansh26 / Attention-Mask-Patterns
  Using FlexAttention to compute attention with different masking patterns
  ☆47Updated last year
• lucidrains / learning-to-expire-pytorch
  An implementation of Transformer with Expire-Span, a circuit for learning which memories to retain
  ☆34Updated 5 years ago
• L1aoXingyu / llm-infer-bench
  ☆12Updated 2 years ago