Alternatives and similar repositories for GoLU

Users that are interested in GoLU are comparing it to the libraries listed below

• MambaMixer / M2
  ☆48Updated last year
• lessw2020 / FAdam_PyTorch
  an implementation of FAdam (Fisher Adam) in PyTorch
  ☆50Updated 7 months ago
• lucidrains / adam-atan2-pytorch
  Implementation of the proposed Adam-atan2 from Google Deepmind in Pytorch
  ☆135Updated 3 months ago
• lucidrains / light-recurrent-unit-pytorch
  Implementation of a Light Recurrent Unit in Pytorch
  ☆49Updated last year
• lucidrains / agent-attention-pytorch
  Implementation of Agent Attention in Pytorch
  ☆93Updated last year
• Optimization-AI / FastCLIP
  Distributed Optimization Infra for learning CLIP models
  ☆27Updated last year
• lucidrains / deep-cross-attention
  Implementation of the proposed DeepCrossAttention by Heddes et al at Google research, in Pytorch
  ☆96Updated 11 months ago
• OscarXZQ / weight-selection
  ☆191Updated last year
• lzw-lzw / UnifiedMLLM
  UnifiedMLLM: Enabling Unified Representation for Multi-modal Multi-tasks With Large Language Model
  ☆22Updated last year
• snu-mllab / LayerMerge
  Official PyTorch implementation of "LayerMerge: Neural Network Depth Compression through Layer Pruning and Merging" (ICML 2024)
  ☆31Updated last year
• RWKV / RWKV-LM
  RWKV is an RNN with transformer-level LLM performance. It can be directly trained like a GPT (parallelizable). So it's combining the best…
  ☆59Updated 10 months ago
• ExplainableML / fomo_in_flux
  Code and benchmark for the paper: "A Practitioner's Guide to Continual Multimodal Pretraining" [NeurIPS'24]
  ☆61Updated last year
• amazon-science / adaptive-feature-transfer
  Official implementation of Adaptive Feature Transfer (AFT)
  ☆23Updated last year
• philippe-eecs / small-vision
  A big_vision inspired repo that implements a generic Auto-Encoder class capable in representation learning and generative modeling.
  ☆34Updated last year
• WailordHe / DenseSSM
  A repository for DenseSSMs
  ☆88Updated last year
• BryceZhuo / PolyCom
  The official implementation of ICLR 2025 paper "Polynomial Composition Activations: Unleashing the Dynamics of Large Language Models".
  ☆18Updated 9 months ago
• lucidrains / infini-transformer-pytorch
  Implementation of Infini-Transformer in Pytorch
  ☆112Updated last year
• kyegomez / Griffin
  Implementation of Griffin from the paper: "Griffin: Mixing Gated Linear Recurrences with Local Attention for Efficient Language Models"
  ☆56Updated 3 months ago
• lucidrains / self-reasoning-tokens-pytorch
  Exploration into the proposed "Self Reasoning Tokens" by Felipe Bonetto
  ☆57Updated last year
• zhixuan-lin / forgetting-transformer
  [ICLR 2025 & COLM 2025] Official PyTorch implementation of the Forgetting Transformer and Adaptive Computation Pruning
  ☆137Updated last month
• lucidrains / mixture-of-attention
  Some personal experiments around routing tokens to different autoregressive attention, akin to mixture-of-experts
  ☆122Updated last year
• AnonymousAlethiometer / SGD_SaI
  Official PyTorch Implementation for Paper "No More Adam: Learning Rate Scaling at Initialization is All You Need"
  ☆55Updated last year
• layer6ai-labs / fusemix
  Data-Efficient Multimodal Fusion on a Single GPU
  ☆68Updated last year
• NX-AI / mlstm_kernels
  Tiled Flash Linear Attention library for fast and efficient mLSTM Kernels.
  ☆84Updated 2 months ago
• ariG23498 / mmdp
  ☆34Updated 7 months ago
• nreHieW / minARImageGen
  Autoregressive Image Generation
  ☆31Updated 7 months ago
• OpenNLPLab / HGRN2
  HGRN2: Gated Linear RNNs with State Expansion
  ☆56Updated last year
• TomerRonen34 / mixed-resolution-vit
  ☆56Updated 2 years ago
• UCDvision / NOLA
  Code for NOLA, an implementation of "nola: Compressing LoRA using Linear Combination of Random Basis"
  ☆57Updated last year
• lucidrains / PEER-pytorch
  Pytorch implementation of the PEER block from the paper, Mixture of A Million Experts, by Xu Owen He at Deepmind
  ☆135Updated 3 months ago