HeegyuKim / torch-xla-SPMD
Pytorch/XLA SPMD Test code in Google TPU
☆21Updated 7 months ago
Related projects ⓘ
Alternatives and complementary repositories for torch-xla-SPMD
- Code for Zero-Shot Tokenizer Transfer☆117Updated last month
- A fast implementation of T5/UL2 in PyTorch using Flash Attention☆71Updated last month
- QLoRA with Enhanced Multi GPU Support☆36Updated last year
- Collection of autoregressive model implementation☆67Updated this week
- ☆31Updated 10 months ago
- Prune transformer layers☆65Updated 5 months ago
- The simplest, fastest repository for training/finetuning medium-sized GPTs.☆84Updated this week
- This is the official repository for Inheritune.☆105Updated last month
- ☆73Updated 4 months ago
- ☆77Updated 5 months ago
- ☆49Updated 8 months ago
- Exploring finetuning public checkpoints on filter 8K sequences on Pile☆115Updated last year
- Language models scale reliably with over-training and on downstream tasks☆94Updated 7 months ago
- A repository for research on medium sized language models.☆74Updated 6 months ago
- This repo is based on https://github.com/jiaweizzhao/GaLore☆19Updated 2 months ago
- Small and Efficient Mathematical Reasoning LLMs☆71Updated 9 months ago
- ☆28Updated 5 months ago
- ☆77Updated 7 months ago
- Reference implementation for Reward-Augmented Decoding: Efficient Controlled Text Generation With a Unidirectional Reward Model☆41Updated 10 months ago
- Implementation of the Llama architecture with RLHF + Q-learning☆157Updated 11 months ago
- Pytorch implementation for "Compressed Context Memory For Online Language Model Interaction" (ICLR'24)☆50Updated 7 months ago
- ☆20Updated last year
- ☆64Updated 4 months ago
- Code repository for the c-BTM paper☆105Updated last year
- Flash Attention Implementation with Multiple Backend Support and Sharding This module provides a flexible implementation of Flash Attenti…☆18Updated last week
- ☆53Updated 3 weeks ago
- One Initialization to Rule them All: Fine-tuning via Explained Variance Adaptation☆30Updated last month
- Yet another random morning idea to be quickly tried and architecture shared if it works; to allow the transformer to pause for any amount…☆49Updated last year
- Anchored Preference Optimization and Contrastive Revisions: Addressing Underspecification in Alignment☆46Updated 2 months ago
- Repository containing the SPIN experiments on the DIBT 10k ranked prompts☆23Updated 8 months ago