Alternatives and similar repositories for pytorch-transformer-distributed:

Users that are interested in pytorch-transformer-distributed are comparing it to the libraries listed below

• hkproj / triton-flash-attention
  ☆128Updated last month
• huggingface / picotron_tutorial
  ☆135Updated last week
• ThinamXx / Meta-llama
  Complete implementation of Llama2 with/without KV cache & inference 🚀
  ☆47Updated 8 months ago
• melisa-writer / short-transformers
  Prune transformer layers
  ☆67Updated 8 months ago
• gpu-mode / profiling-cuda-in-torch
  ☆142Updated last year
• hkproj / dpo-notes
  Notes on Direct Preference Optimization
  ☆16Updated 10 months ago
• rasbt / dora-from-scratch
  LoRA and DoRA from Scratch Implementations
  ☆196Updated 11 months ago
• hkproj / pytorch-llama
  LLaMA 2 implemented from scratch in PyTorch
  ☆294Updated last year
• hkproj / rlhf-ppo
  Notes and commented code for RLHF (PPO)
  ☆69Updated 11 months ago
• golololologol / LLM-Distillery
  A pipeline for LLM knowledge distillation
  ☆91Updated 3 weeks ago
• joey00072 / ohara
  Collection of autoregressive model implementation
  ☆81Updated last week
• hkproj / pytorch-llama-notes
  Notes about LLaMA 2 model
  ☆53Updated last year
• alperiox / Compact-Language-Models-via-Pruning-and-Knowledge-Distillation
  Unofficial implementation of https://arxiv.org/pdf/2407.14679
  ☆42Updated 5 months ago
• gpu-mode / ring-attention
  ring-attention experiments
  ☆123Updated 4 months ago
• ThinamXx / build-GPT
  Building GPT ...
  ☆17Updated 2 months ago
• nyunAI / Faster-LLM-Survey
  ☆40Updated 9 months ago
• kyegomez / Infini-attention
  Implementation of the paper: "Leave No Context Behind: Efficient Infinite Context Transformers with Infini-attention" from Google in pyTO…
  ☆53Updated last week
• AviSoori1x / seemore
  From scratch implementation of a vision language model in pure PyTorch
  ☆194Updated 9 months ago
• itsnamgyu / block-transformer
  Block Transformer: Global-to-Local Language Modeling for Fast Inference (NeurIPS 2024)
  ☆149Updated 2 months ago
• hkproj / pytorch-lora
  LORA: Low-Rank Adaptation of Large Language Models implemented using PyTorch
  ☆94Updated last year
• facebookresearch / LayerSkip
  Code for "LayerSkip: Enabling Early Exit Inference and Self-Speculative Decoding", ACL 2024
  ☆272Updated last week
• tanaymeh / mamba-train
  A single repo with all scripts and utils to train / fine-tune the Mamba model with or without FIM
  ☆51Updated 10 months ago
• Pleias / Various-Finetuning
  Set of scripts to finetune LLMs
  ☆36Updated 10 months ago
• SeunghyunSEO / optimized_hf_llama_class_for_training
  ☆47Updated 5 months ago
• AIMO-CMU-MATH / CMU_MATH-AIMO
  ☆70Updated 7 months ago
• cindysridykhan / instruct_storyteller_tinyllama2
  Training and Fine-tuning an llm in Python and PyTorch.
  ☆41Updated last year
• NVlabs / hymba
  ☆167Updated 2 months ago