Alternatives and similar repositories for transformer-lm-gan:

Users that are interested in transformer-lm-gan are comparing it to the libraries listed below

• lucidrains / self-reasoning-tokens-pytorch
  Exploration into the proposed "Self Reasoning Tokens" by Felipe Bonetto
  ☆55Updated 10 months ago
• BlinkDL / LinearAttentionArena
  Here we will test various linear attention designs.
  ☆60Updated 11 months ago
• foundation-model-stack / bamba
  Train, tune, and infer Bamba model
  ☆88Updated 2 months ago
• lucidrains / light-recurrent-unit-pytorch
  Implementation of a Light Recurrent Unit in Pytorch
  ☆47Updated 6 months ago
• Zyphra / Zyda_processing
  ☆33Updated 9 months ago
• shreyansh26 / Attention-Mask-Patterns
  Using FlexAttention to compute attention with different masking patterns
  ☆43Updated 6 months ago
• frankxwang / dpo-prefix-sharing
  DPO, but faster 🚀
  ☆40Updated 4 months ago
• kyleliang919 / Online-Subspace-Descent
  This repo is based on https://github.com/jiaweizzhao/GaLore
  ☆26Updated 6 months ago
• lucidrains / infini-transformer-pytorch
  Implementation of Infini-Transformer in Pytorch
  ☆110Updated 3 months ago
• lucidrains / GAF-microbatch-pytorch
  Implementation of Gradient Agreement Filtering, from Chaubard et al. of Stanford, but for single machine microbatches, in Pytorch
  ☆23Updated 2 months ago
• zhixuan-lin / forgetting-transformer
  [ICLR 2025] Official PyTorch implementation of "Forgetting Transformer: Softmax Attention with a Forget Gate"
  ☆90Updated this week
• nreHieW / minVAR
  Minimal Implementation of Visual Autoregressive Modelling (VAR)
  ☆29Updated 3 weeks ago
• amirzandieh / HyperAttention
  Triton Implementation of HyperAttention Algorithm
  ☆47Updated last year
• LegallyCoder / mamba-hf
  Implementation of the Mamba SSM with hf_integration.
  ☆56Updated 7 months ago
• SprocketLab / sparse_matrix_fine_tuning
  Official repository for ICML 2024 paper "MoRe Fine-Tuning with 10x Fewer Parameters"
  ☆17Updated last month
• RWKV / ZeroCoT
  https://x.com/BlinkDL_AI/status/1884768989743882276
  ☆27Updated 2 months ago
• penfever / wildchat-50m
  Code, results and other artifacts from the paper introducing the WildChat-50m dataset and the Re-Wild model family.
  ☆29Updated 2 weeks ago
• fal-ai-community / nano-mdm
  Tiny re-implementation of MDM in style of LLaDA and nano-gpt speedrun
  ☆46Updated last month
• sustcsonglin / gated_linear_attention_layer
  ☆32Updated last year
• RobertCsordas / moe
  Official repository for the paper "Approximating Two-Layer Feedforward Networks for Efficient Transformers"
  ☆36Updated last year
• evanatyourservice / llm-jax
  Train a SmolLM-style llm on fineweb-edu in JAX/Flax with an assortment of optimizers.
  ☆17Updated 3 weeks ago
• lucidrains / memory-editable-transformer
  My explorations into editing the knowledge and memories of an attention network
  ☆34Updated 2 years ago
• glassroom / heinsen_attention
  Reference implementation of "Softmax Attention with Constant Cost per Token" (Heinsen, 2024)
  ☆24Updated 10 months ago
• wang-kee / LiNeS
  Official repository of "LiNeS: Post-training Layer Scaling Prevents Forgetting and Enhances Model Merging"
  ☆24Updated 5 months ago
• lucidrains / quartic-transformer
  Exploring an idea where one forgets about efficiency and carries out attention across each edge of the nodes (tokens)
  ☆50Updated 2 weeks ago
• kaiokendev / cutoff-len-is-context-len
  Demonstration that finetuning RoPE model on larger sequences than the pre-trained model adapts the model context limit
  ☆63Updated last year
• jongwooko / distillm-2
  ☆9Updated last month
• lucaslingle / mu_transformer
  Transformer with Mu-Parameterization, implemented in Jax/Flax. Supports FSDP on TPU pods.
  ☆30Updated 4 months ago
• yikangshen / megablocks
  ☆19Updated 10 months ago
• lucidrains / mixture-of-attention
  Some personal experiments around routing tokens to different autoregressive attention, akin to mixture-of-experts
  ☆118Updated 5 months ago