Alternatives and similar repositories for DifferentialTransformer

Users that are interested in DifferentialTransformer are comparing it to the libraries listed below

• lucidrains / minGRU-pytorch
  Implementation of the proposed minGRU in Pytorch
  ☆300Updated 4 months ago
• kyegomez / Griffin
  Implementation of Griffin from the paper: "Griffin: Mixing Gated Linear Recurrences with Local Attention for Efficient Language Models"
  ☆55Updated this week
• myscience / x-lstm
  Pytorch implementation of the xLSTM model by Beck et al. (2024)
  ☆169Updated 11 months ago
• knotgrass / attention
  several types of attention modules written in PyTorch for learning purposes
  ☆53Updated 9 months ago
• andrewgcodes / xlstm
  my attempts at implementing various bits of Sepp Hochreiter's new xLSTM architecture
  ☆130Updated last year
• CG80499 / KAN-GPT-2
  Training small GPT-2 style models using Kolmogorov-Arnold networks.
  ☆120Updated last year
• SynodicMonth / ChebyKAN
  Kolmogorov-Arnold Networks (KAN) using Chebyshev polynomials instead of B-splines.
  ☆381Updated last year
• kyegomez / xLSTM
  Implementation of xLSTM in Pytorch from the paper: "xLSTM: Extended Long Short-Term Memory"
  ☆119Updated this week
• 1ssb / torchkan
  An easy to use PyTorch implementation of the Kolmogorov Arnold Network and a few novel variations
  ☆184Updated 8 months ago
• NX-AI / mlstm_kernels
  Tiled Flash Linear Attention library for fast and efficient mLSTM Kernels.
  ☆64Updated 3 weeks ago
• fkodom / yet-another-retnet
  A simple but robust PyTorch implementation of RetNet from "Retentive Network: A Successor to Transformer for Large Language Models" (http…
  ☆106Updated last year
• Jaykef / ai-algorithms
  First-principle implementations of groundbreaking AI algorithms using a wide range of deep learning frameworks, accompanied by supporting…
  ☆177Updated this week
• AI-Guru / helibrunna
  A HuggingFace compatible Small Language Model trainer.
  ☆75Updated 5 months ago
• NVIDIA / logits-processor-zoo
  A collection of LogitsProcessors to customize and enhance LLM behavior for specific tasks.
  ☆323Updated 2 weeks ago
• lucidrains / adam-atan2-pytorch
  Implementation of the proposed Adam-atan2 from Google Deepmind in Pytorch
  ☆110Updated 7 months ago
• bobby-he / simplified_transformers
  ☆292Updated 7 months ago
• lucidrains / infini-transformer-pytorch
  Implementation of Infini-Transformer in Pytorch
  ☆111Updated 6 months ago
• kyegomez / MambaTransformer
  Integrating Mamba/SSMs with Transformer for Enhanced Long Context and High-Quality Sequence Modeling
  ☆197Updated this week
• lucidrains / block-recurrent-transformer-pytorch
  Implementation of Block Recurrent Transformer - Pytorch
  ☆221Updated 11 months ago
• lucidrains / gradnorm-pytorch
  A practical implementation of GradNorm, Gradient Normalization for Adaptive Loss Balancing, in Pytorch
  ☆99Updated last year
• tascj / kaggle-lmsys-chatbot-arena
  Solution of Kaggle competition: LMSYS - Chatbot Arena Human Preference Predictions
  ☆44Updated 11 months ago
• AI-Guru / xlstm-resources
  Resources about xLSTM by Sepp Hochreiter
  ☆318Updated 8 months ago
• lucidrains / agent-attention-pytorch
  Implementation of Agent Attention in Pytorch
  ☆91Updated last year
• google-deepmind / randomized_positional_encodings
  Randomized Positional Encodings Boost Length Generalization of Transformers
  ☆82Updated last year
• nikhilvyas / SOAP
  ☆205Updated 7 months ago
• Lgeu / santa2024
  ☆48Updated 5 months ago
• myscience / mamba
  Pytorch (Lightning) implementation of the Mamba model
  ☆29Updated 3 months ago
• joey00072 / nanoGRPO
  nanoGRPO is a lightweight implementation of Group Relative Policy Optimization (GRPO)
  ☆111Updated 2 months ago
• wolfecameron / nanoMoE
  An extension of the nanoGPT repository for training small MOE models.
  ☆163Updated 4 months ago
• PeaBrane / mamba-tiny
  Simple, minimal implementation of the Mamba SSM in one pytorch file. Using logcumsumexp (Heisen sequence).
  ☆120Updated 9 months ago