HeegyuKim / torch-xla-SPMD

Related projects ⓘ

Alternatives and complementary repositories for torch-xla-SPMD

• cloneofsimo / fim-llama-deepspeed
  ☆31Updated 10 months ago
• catie-aq / flashT5
  A fast implementation of T5/UL2 in PyTorch using Flash Attention
  ☆67Updated 3 weeks ago
• lucidrains / pause-transformer
  Yet another random morning idea to be quickly tried and architecture shared if it works; to allow the transformer to pause for any amount…
  ☆49Updated last year
• kyleliang919 / Long-context-transformers
  Exploring finetuning public checkpoints on filter 8K sequences on Pile
  ☆115Updated last year
• r-three / RAD
  Reference implementation for Reward-Augmented Decoding: Efficient Controlled Text Generation With a Unidirectional Reward Model
  ☆41Updated 9 months ago
• TristanThrush / i-am-a-strange-dataset
  Repository for "I am a Strange Dataset: Metalinguistic Tests for Language Models"
  ☆39Updated 10 months ago
• bminixhofer / zett
  Code for Zero-Shot Tokenizer Transfer
  ☆115Updated 3 weeks ago
• joey00072 / ohara
  Collection of autoregressive model implementation
  ☆66Updated last week
• mgmalek / efficient_cross_entropy
  ☆76Updated 5 months ago
• minyoungg / LTE
  ☆63Updated 4 months ago
• mlfoundations / scaling
  Language models scale reliably with over-training and on downstream tasks
  ☆94Updated 7 months ago
• melisa-writer / short-transformers
  Prune transformer layers
  ☆64Updated 5 months ago
• euclaise / supertrainer2000
  ☆49Updated 7 months ago
• microsoft / mutransformers
  some common Huggingface transformers in maximal update parametrization (µP)
  ☆76Updated 2 years ago
• kaistAI / LangBridge
  [ACL 2024] LangBridge: Multilingual Reasoning Without Multilingual Supervision
  ☆78Updated last week
• kernelmachine / cbtm
  Code repository for the c-BTM paper
  ☆105Updated last year
• imoneoi / multipack_sampler
  Multipack distributed sampler for fast padding-free training of LLMs
  ☆176Updated 3 months ago
• cloneofsimo / min-max-gpt
  Minimal (400 LOC) implementation Maximum (multi-node, FSDP) GPT training
  ☆112Updated 6 months ago
• schwartz-lab-NLP / TOVA
  Token Omission Via Attention
  ☆119Updated 3 weeks ago
• siyan-zhao / prepacking
  The source code of our work "Prepacking: A Simple Method for Fast Prefilling and Increased Throughput in Large Language Models"
  ☆56Updated last month
• EleutherAI / nanoGPT-mup
  The simplest, fastest repository for training/finetuning medium-sized GPTs.
  ☆84Updated last week
• ChrisHayduk / qlora-multi-gpu
  QLoRA with Enhanced Multi GPU Support
  ☆36Updated last year
• Edward-Sun / gpt-accelera
  Simple and efficient pytorch-native transformer training and inference (batched)
  ☆61Updated 7 months ago
• TRI-ML / linear_open_lm
  A repository for research on medium sized language models.
  ☆74Updated 5 months ago
• HanGuo97 / lq-lora
  ☆121Updated 9 months ago
• cloneofsimo / min-fsdp
  ☆72Updated 4 months ago
• facebookresearch / mexma
  MEXMA: Token-level objectives improve sentence representations
  ☆32Updated this week
• snu-mllab / Context-Memory
  Pytorch implementation for "Compressed Context Memory For Online Language Model Interaction" (ICLR'24)
  ☆49Updated 6 months ago
• epfml / schedules-and-scaling
  ☆50Updated last week
• Rallio67 / language-model-agents
  Experiments with generating opensource language model assistants
  ☆97Updated last year