Alternatives and similar repositories for Mega-pytorch

Users that are interested in Mega-pytorch are comparing it to the libraries listed below

• lucidrains / gated-state-spaces-pytorch
  Implementation of Gated State Spaces, from the paper "Long Range Language Modeling via Gated State Spaces", in Pytorch
  ☆101Updated 2 years ago
• ctlllll / SGConv
  ☆164Updated 2 years ago
• lucidrains / flash-cosine-sim-attention
  Implementation of fused cosine similarity attention in the same style as Flash Attention
  ☆214Updated 2 years ago
• facebookresearch / mega
  Sequence modeling with Mega.
  ☆300Updated 2 years ago
• lucidrains / ponder-transformer
  Implementation of a Transformer that Ponders, using the scheme from the PonderNet paper
  ☆81Updated 3 years ago
• lucidrains / block-recurrent-transformer-pytorch
  Implementation of Block Recurrent Transformer - Pytorch
  ☆220Updated last year
• lucidrains / discrete-key-value-bottleneck-pytorch
  Implementation of Discrete Key / Value Bottleneck, in Pytorch
  ☆88Updated 2 years ago
• lucidrains / fast-transformer-pytorch
  Implementation of Fast Transformer in Pytorch
  ☆175Updated 4 years ago
• lucidrains / Adan-pytorch
  Implementation of the Adan (ADAptive Nesterov momentum algorithm) Optimizer in Pytorch
  ☆251Updated 3 years ago
• jiaweizzhao / ZerO-initialization
  ☆75Updated 2 years ago
• ag1988 / dss
  Sequence Modeling with Structured State Spaces
  ☆66Updated 3 years ago
• lucidrains / nystrom-attention
  Implementation of Nyström Self-attention, from the paper Nyströmformer
  ☆139Updated 5 months ago
• lucidrains / compressive-transformer-pytorch
  Pytorch implementation of Compressive Transformers, from Deepmind
  ☆164Updated 3 years ago
• lucidrains / long-short-transformer
  Implementation of Long-Short Transformer, combining local and global inductive biases for attention over long sequences, in Pytorch
  ☆120Updated 4 years ago
• lucidrains / axial-positional-embedding
  Axial Positional Embedding for Pytorch
  ☆83Updated 6 months ago
• lucidrains / product-key-memory
  Standalone Product Key Memory module in Pytorch - for augmenting Transformer models
  ☆82Updated last year
• lucidrains / perceiver-ar-pytorch
  Implementation of Perceiver AR, Deepmind's new long-context attention network based on Perceiver architecture, in Pytorch
  ☆91Updated 2 years ago
• lucidrains / CoLT5-attention
  Implementation of the conditionally routed attention in the CoLT5 architecture, in Pytorch
  ☆229Updated last year
• AminRezaei0x443 / memory-efficient-attention
  Memory Efficient Attention (O(sqrt(n)) for Jax and PyTorch
  ☆184Updated 2 years ago
• lucidrains / feedback-transformer-pytorch
  Implementation of Feedback Transformer in Pytorch
  ☆107Updated 4 years ago
• lucidrains / simple-hierarchical-transformer
  Experiments around a simple idea for inducing multiple hierarchical predictive model within a GPT
  ☆220Updated last year
• lucidrains / gateloop-transformer
  Implementation of GateLoop Transformer in Pytorch and Jax
  ☆90Updated last year
• lucidrains / h-transformer-1d
  Implementation of H-Transformer-1D, Hierarchical Attention for Sequence Learning
  ☆164Updated last year
• lucidrains / memformer
  Implementation of Memformer, a Memory-augmented Transformer, in Pytorch
  ☆123Updated 4 years ago
• lucidrains / hourglass-transformer-pytorch
  Implementation of Hourglass Transformer, in Pytorch, from Google and OpenAI
  ☆94Updated 3 years ago
• lucidrains / memory-editable-transformer
  My explorations into editing the knowledge and memories of an attention network
  ☆35Updated 2 years ago
• OATML / RHO-Loss
  ☆208Updated 2 years ago
• lucidrains / panoptic-transformer
  Another attempt at a long-context / efficient transformer by me
  ☆38Updated 3 years ago
• infocusp / diffusion_models
  Minimal standalone example of diffusion model
  ☆159Updated 3 years ago
• lucidrains / g-mlp-gpt
  GPT, but made only out of MLPs
  ☆89Updated 4 years ago