Alternatives and similar repositories for Transformer-Workbench

Users that are interested in Transformer-Workbench are comparing it to the libraries listed below

• knotgrass / attention
  several types of attention modules written in PyTorch for learning purposes
  ☆52Updated last year
• YeonwooSung / Pytorch_mixture-of-experts
  PyTorch implementation of moe, which stands for mixture of experts
  ☆50Updated 4 years ago
• RobertCsordas / moe_attention
  Official repository for the paper "SwitchHead: Accelerating Transformers with Mixture-of-Experts Attention"
  ☆100Updated last year
• ShaderManager / RetNet
  PyTorch implementation of Retentive Network: A Successor to Transformer for Large Language Models
  ☆14Updated 2 years ago
• SeunghyunSEO / optimized_hf_llama_class_for_training
  ☆48Updated last year
• fkodom / grouped-query-attention-pytorch
  (Unofficial) PyTorch implementation of grouped-query attention (GQA) from "GQA: Training Generalized Multi-Query Transformer Models from …
  ☆181Updated last year
• Jaykef / ai-algorithms
  First-principle implementations of groundbreaking AI algorithms using a wide range of deep learning frameworks, accompanied by supporting…
  ☆178Updated 3 months ago
• RustamyF / clip-multimodal-ml
  ☆67Updated last year
• jlamprou / Infini-Attention
  Efficient Infinite Context Transformers with Infini-attention Pytorch Implementation + QwenMoE Implementation + Training Script + 1M cont…
  ☆84Updated last year
• geronimi73 / 3090_shorts
  minimal scripts for 24GB VRAM GPUs. training, inference, whatever
  ☆43Updated this week
• The-AI-Summer / pytorch-ddp
  code for the ddp tutorial
  ☆32Updated 3 years ago
• WalkerMitty / Fast-Llama2
  Fast instruction tuning with Llama2
  ☆11Updated last year
• tigerchen52 / awesome_role_of_small_models
  a curated list of the role of small models in the LLM era
  ☆107Updated last year
• lucidrains / CALM-pytorch
  Implementation of CALM from the paper "LLM Augmented LLMs: Expanding Capabilities through Composition", out of Google Deepmind
  ☆177Updated last year
• wjbmattingly / qwen2-vl-finetune-huggingface
  This project is a collection of fine-tuning scripts to help researchers fine-tune Qwen 2 VL on HuggingFace datasets.
  ☆74Updated 3 months ago
• nyunAI / Faster-LLM-Survey
  ☆42Updated last year
• rasbt / cvpr2023
  ☆134Updated 2 years ago
• rasbt / dora-from-scratch
  LoRA and DoRA from Scratch Implementations
  ☆211Updated last year
• fkodom / soft-mixture-of-experts
  PyTorch implementation of Soft MoE by Google Brain in "From Sparse to Soft Mixtures of Experts" (https://arxiv.org/pdf/2308.00951.pdf)
  ☆78Updated 2 years ago
• juvi21 / CoPE-cuda
  Contextual Position Encoding but with some custom CUDA Kernels https://arxiv.org/abs/2405.18719
  ☆22Updated last year
• daniel-furman / sft-demos
  Lightweight demos for finetuning LLMs. Powered by 🤗 transformers and open-source datasets.
  ☆78Updated last year
• Mr-Tieguigui / LLM-Post-Training
  ☆68Updated 5 months ago
• YuchuanTian / RethinkTinyLM
  [ICML'24] The official implementation of “Rethinking Optimization and Architecture for Tiny Language Models”
  ☆124Updated 9 months ago
• fshnkarimi / Fine-tuning-an-LLM-using-LoRA
  📚 Text Classification with LoRA (Low-Rank Adaptation) of Language Models - Efficiently fine-tune large language models for text classifi…
  ☆54Updated 2 years ago
• jiahe7ay / infini-mini-transformer
  This is a personal reimplementation of Google's Infini-transformer, utilizing a small 2b model. The project includes both model and train…
  ☆58Updated last year
• kyegomez / Griffin
  Implementation of Griffin from the paper: "Griffin: Mixing Gated Linear Recurrences with Local Attention for Efficient Language Models"
  ☆56Updated last week
• kyegomez / TTL
  Pytorch Implementation of the paper: "Learning to (Learn at Test Time): RNNs with Expressive Hidden States"
  ☆25Updated this week
• Hemanthkumar2112 / Reward-Modeling-RLHF-Finetune-and-RAG
  Gemma2(9B), Llama3-8B-Finetune-and-RAG, code base for sample, implemented in Kaggle platform
  ☆22Updated 8 months ago
• mingyin0312 / RL4LLM
  RL significantly the reasoning capability of Qwen2.5-1.5B-Instruct
  ☆30Updated 8 months ago
• thepowerfuldeez / OLMo
  My fork os allen AI's OLMo for educational purposes.
  ☆30Updated 10 months ago