Alternatives and similar repositories for transformer-xl:

Users that are interested in transformer-xl are comparing it to the libraries listed below

• dvruette / barrel-rec-pytorch
  ☆53Updated last year
• epfml / DenseFormer
  ☆79Updated 11 months ago
• young-geng / mintext
  Minimal but scalable implementation of large language models in JAX
  ☆34Updated 4 months ago
• HomebrewML / Olmax
  HomebrewNLP in JAX flavour for maintable TPU-Training
  ☆49Updated last year
• lucidrains / pause-transformer
  Yet another random morning idea to be quickly tried and architecture shared if it works; to allow the transformer to pause for any amount…
  ☆53Updated last year
• NohTow / PPL-MCTS
  Repository for the code of the "PPL-MCTS: Constrained Textual Generation Through Discriminator-Guided Decoding" paper, NAACL'22
  ☆64Updated 2 years ago
• RobertCsordas / moe
  Official repository for the paper "Approximating Two-Layer Feedforward Networks for Efficient Transformers"
  ☆36Updated last year
• irhum / hyena
  JAX/Flax implementation of the Hyena Hierarchy
  ☆34Updated last year
• microsoft / mutransformers
  some common Huggingface transformers in maximal update parametrization (µP)
  ☆80Updated 3 years ago
• McGill-NLP / length-generalization
  Code for the paper "The Impact of Positional Encoding on Length Generalization in Transformers", NeurIPS 2023
  ☆131Updated 11 months ago
• lucidrains / gateloop-transformer
  Implementation of GateLoop Transformer in Pytorch and Jax
  ☆87Updated 9 months ago
• likenneth / q_probe
  Q-Probe: A Lightweight Approach to Reward Maximization for Language Models
  ☆41Updated 9 months ago
• lucidrains / GAF-microbatch-pytorch
  Implementation of Gradient Agreement Filtering, from Chaubard et al. of Stanford, but for single machine microbatches, in Pytorch
  ☆23Updated 2 months ago
• lucidrains / token-shift-gpt
  Implementation of Token Shift GPT - An autoregressive model that solely relies on shifting the sequence space for mixing
  ☆48Updated 3 years ago
• cloneofsimo / zeroshampoo
  ☆33Updated 6 months ago
• facebookresearch / oni
  Learn online intrinsic rewards from LLM feedback
  ☆35Updated 3 months ago
• Sea-Snell / JAXSeq
  Train very large language models in Jax.
  ☆203Updated last year
• mlfoundations / scaling
  Language models scale reliably with over-training and on downstream tasks
  ☆96Updated 11 months ago
• cloneofsimo / min-fsdp
  ☆76Updated 8 months ago
• sholtodouglas / scalingExperiments
  ☆60Updated 3 years ago
• hamishivi / EasyLM
  Large language models (LLMs) made easy, EasyLM is a one stop solution for pre-training, finetuning, evaluating and serving LLMs in JAX/Fl…
  ☆72Updated 7 months ago
• conglu1997 / intelligent-go-explore
  Intelligent Go-Explore: Standing on the Shoulders of Giant Foundation Models
  ☆54Updated last month
• BlinkDL / SmallInitEmb
  LayerNorm(SmallInit(Embedding)) in a Transformer to improve convergence
  ☆60Updated 3 years ago
• google-research-datasets / QAmeleon
  QAmeleon introduces synthetic multilingual QA data using PaLM, a 540B large language model. This dataset was generated by prompt tuning P…
  ☆34Updated last year
• lucaslingle / mu_transformer
  Transformer with Mu-Parameterization, implemented in Jax/Flax. Supports FSDP on TPU pods.
  ☆30Updated 3 months ago
• euclaise / supertrainer2000
  ☆49Updated last year
• lucidrains / quartic-transformer
  Exploring an idea where one forgets about efficiency and carries out attention across each edge of the nodes (tokens)
  ☆47Updated this week
• edwardmilsom / function-space-learning-rates-paper
  Code for the paper "Function-Space Learning Rates"
  ☆17Updated last month
• CarperAI / Algorithm-Distillation-RLHF
  ☆34Updated 2 years ago
• young-geng / tpu_pod_commander
  TPU pod commander is a package for managing and launching jobs on Google Cloud TPU pods.
  ☆20Updated 9 months ago