frankxwang / dpo-prefix-sharing
DPO, but faster π
β23Updated 3 weeks ago
Related projects β
Alternatives and complementary repositories for dpo-prefix-sharing
- β27Updated 5 months ago
- Using FlexAttention to compute attention with different masking patternsβ40Updated last month
- A repository for research on medium sized language models.β74Updated 5 months ago
- β62Updated 3 months ago
- A public implementation of the ReLoRA pretraining method, built on Lightning-AI's Pytorch Lightning suite.β33Updated 8 months ago
- Implementation of the paper: "Leave No Context Behind: Efficient Infinite Context Transformers with Infini-attention" from Google in pyTOβ¦β52Updated last week
- Triton Implementation of HyperAttention Algorithmβ46Updated 11 months ago
- Official repository for the paper "Approximating Two-Layer Feedforward Networks for Efficient Transformers"β36Updated last year
- β22Updated 2 weeks ago
- Official implementation of ECCV24 paper: POAβ24Updated 3 months ago
- Anchored Preference Optimization and Contrastive Revisions: Addressing Underspecification in Alignmentβ46Updated 2 months ago
- Collection of autoregressive model implementationβ67Updated this week
- GoldFinch and other hybrid transformer componentsβ39Updated 4 months ago
- β45Updated 2 months ago
- β48Updated last month
- Official repository for the paper "SwitchHead: Accelerating Transformers with Mixture-of-Experts Attention"β92Updated last month
- β35Updated 9 months ago
- The official implementation of MARS: Unleashing the Power of Variance Reduction for Training Large Modelsβ65Updated this week
- This repo is based on https://github.com/jiaweizzhao/GaLoreβ19Updated 2 months ago
- Code for the arXiv preprint "The Unreasonable Effectiveness of Easy Training Data"β44Updated 10 months ago
- Code for Paper: Harnessing Webpage Uis For Text Rich Visual Understandingβ38Updated last month
- Code for the examples presented in the talk "Training a Llama in your backyard: fine-tuning very large models on consumer hardware" givenβ¦β14Updated last year
- β43Updated 2 months ago
- Repository for Sparse Finetuning of LLMs via modified version of the MosaicML llmfoundryβ38Updated 10 months ago
- β28Updated 7 months ago
- Linear Attention Sequence Parallelism (LASP)β64Updated 5 months ago
- Code for "Your Mixture-of-Experts LLM Is Secretly an Embedding Model For Free"β36Updated last month
- Lottery Ticket Adaptationβ36Updated last month
- Script for processing OpenAI's PRM800K process supervision dataset into an Alpaca-style instruction-response formatβ27Updated last year
- Codebase for Instruction Following without Instruction Tuningβ31Updated last month