Alternatives and similar repositories for pytorch-transformer-distributed:

Users that are interested in pytorch-transformer-distributed are comparing it to the libraries listed below

• melisa-writer / short-transformers
  Prune transformer layers
  ☆68Updated 9 months ago
• hkproj / triton-flash-attention
  ☆136Updated 2 months ago
• hkproj / dpo-notes
  Notes on Direct Preference Optimization
  ☆18Updated 11 months ago
• huggingface / picotron_tutorial
  ☆158Updated last month
• ThinamXx / Meta-llama
  Complete implementation of Llama2 with/without KV cache & inference 🚀
  ☆47Updated 10 months ago
• nyunAI / Faster-LLM-Survey
  ☆41Updated 11 months ago
• hkproj / quantization-notes
  Notes on quantization in neural networks
  ☆77Updated last year
• CASE-Lab-UMD / LLM-Drop
  The official implementation of the paper "What Matters in Transformers? Not All Attention is Needed".
  ☆164Updated 3 months ago
• hkproj / rlhf-ppo
  Notes and commented code for RLHF (PPO)
  ☆77Updated last year
• rasbt / dora-from-scratch
  LoRA and DoRA from Scratch Implementations
  ☆198Updated last year
• facebookresearch / LayerSkip
  Code for "LayerSkip: Enabling Early Exit Inference and Self-Speculative Decoding", ACL 2024
  ☆277Updated 3 weeks ago
• alperiox / Compact-Language-Models-via-Pruning-and-Knowledge-Distillation
  Unofficial implementation of https://arxiv.org/pdf/2407.14679
  ☆44Updated 6 months ago
• tanaymeh / mamba-train
  A single repo with all scripts and utils to train / fine-tune the Mamba model with or without FIM
  ☆54Updated 11 months ago
• AviSoori1x / seemore
  From scratch implementation of a vision language model in pure PyTorch
  ☆205Updated 10 months ago
• hkproj / pytorch-lora
  LORA: Low-Rank Adaptation of Large Language Models implemented using PyTorch
  ☆99Updated last year
• NVlabs / hymba
  ☆173Updated 3 months ago
• hkproj / pytorch-llama
  LLaMA 2 implemented from scratch in PyTorch
  ☆307Updated last year
• gpu-mode / profiling-cuda-in-torch
  ☆151Updated last year
• neubig / minllama-assignment
  ☆82Updated 5 months ago
• astramind-ai / Mixture-of-depths
  Unofficial implementation for the paper "Mixture-of-Depths: Dynamically allocating compute in transformer-based language models"
  ☆154Updated 9 months ago
• foundation-model-stack / bamba
  Train, tune, and infer Bamba model
  ☆86Updated 2 months ago
• itsnamgyu / block-transformer
  Block Transformer: Global-to-Local Language Modeling for Fast Inference (NeurIPS 2024)
  ☆150Updated 3 months ago
• joey00072 / ohara
  Collection of autoregressive model implementation
  ☆83Updated last month
• mengxiayu / LLMSuperWeight
  Code for studying the super weight in LLM
  ☆91Updated 3 months ago