Alternatives and similar repositories for Transformer-Workbench

Users that are interested in Transformer-Workbench are comparing it to the libraries listed below

• knotgrass / attention
  several types of attention modules written in PyTorch for learning purposes
  ☆52Updated last year
• rasbt / dora-from-scratch
  LoRA and DoRA from Scratch Implementations
  ☆215Updated last year
• YeonwooSung / Pytorch_mixture-of-experts
  PyTorch implementation of moe, which stands for mixture of experts
  ☆51Updated 4 years ago
• jlamprou / Infini-Attention
  Efficient Infinite Context Transformers with Infini-attention Pytorch Implementation + QwenMoE Implementation + Training Script + 1M cont…
  ☆83Updated last year
• RobertCsordas / moe_attention
  Official repository for the paper "SwitchHead: Accelerating Transformers with Mixture-of-Experts Attention"
  ☆102Updated last year
• alperiox / Compact-Language-Models-via-Pruning-and-Knowledge-Distillation
  Unofficial implementation of https://arxiv.org/pdf/2407.14679
  ☆51Updated last year
• fkodom / grouped-query-attention-pytorch
  (Unofficial) PyTorch implementation of grouped-query attention (GQA) from "GQA: Training Generalized Multi-Query Transformer Models from …
  ☆185Updated last year
• fangyuan-ksgk / Mini-LLaVA
  A minimal implementation of LLaVA-style VLM with interleaved image & text & video processing ability.
  ☆97Updated 11 months ago
• lucidrains / CALM-pytorch
  Implementation of CALM from the paper "LLM Augmented LLMs: Expanding Capabilities through Composition", out of Google Deepmind
  ☆179Updated last year
• fangyuan-ksgk / Tiny-GRPO
  minimal GRPO implementation from scratch
  ☆100Updated 9 months ago
• RustamyF / clip-multimodal-ml
  ☆68Updated last year
• fshnkarimi / Fine-tuning-an-LLM-using-LoRA
  📚 Text Classification with LoRA (Low-Rank Adaptation) of Language Models - Efficiently fine-tune large language models for text classifi…
  ☆55Updated 2 years ago
• bzhangGo / rmsnorm
  Root Mean Square Layer Normalization
  ☆258Updated 2 years ago
• kyegomez / TTL
  Pytorch Implementation of the paper: "Learning to (Learn at Test Time): RNNs with Expressive Hidden States"
  ☆25Updated last week
• hkproj / pytorch-transformer-distributed
  Distributed training (multi-node) of a Transformer model
  ☆89Updated last year
• catid / dora
  Implementation of DoRA
  ☆307Updated last year
• aju22 / LLaMA2
  This repository contains an implementation of the LLaMA 2 (Large Language Model Meta AI) model, a Generative Pretrained Transformer (GPT)…
  ☆74Updated 2 years ago
• Nicolas-BZRD / llm-recipes
  ☆29Updated last year
• lucidrains / mixture-of-attention
  Some personal experiments around routing tokens to different autoregressive attention, akin to mixture-of-experts
  ☆121Updated last year
• WalkerMitty / Fast-Llama2
  Fast instruction tuning with Llama2
  ☆11Updated last year
• tigerchen52 / awesome_role_of_small_models
  a curated list of the role of small models in the LLM era
  ☆110Updated last year
• rasbt / pytorch-memory-optim
  This code repository contains the code used for my "Optimizing Memory Usage for Training LLMs and Vision Transformers in PyTorch" blog po…
  ☆92Updated 2 years ago
• FareedKhan-dev / train-llama4
  Building LLaMA 4 MoE from Scratch
  ☆68Updated 7 months ago
• fkodom / soft-mixture-of-experts
  PyTorch implementation of Soft MoE by Google Brain in "From Sparse to Soft Mixtures of Experts" (https://arxiv.org/pdf/2308.00951.pdf)
  ☆79Updated 2 years ago
• nyunAI / Faster-LLM-Survey
  ☆42Updated last year
• The-AI-Summer / pytorch-ddp
  code for the ddp tutorial
  ☆32Updated 3 years ago
• itsnamgyu / block-transformer
  Block Transformer: Global-to-Local Language Modeling for Fast Inference (NeurIPS 2024)
  ☆162Updated 8 months ago
• YuchuanTian / RethinkTinyLM
  [ICML'24] The official implementation of “Rethinking Optimization and Architecture for Tiny Language Models”
  ☆126Updated 11 months ago
• Zefty / rag-end2end-retriever
  ☆20Updated 4 years ago
• SeunghyunSEO / optimized_hf_llama_class_for_training
  ☆48Updated last year