Alternatives and similar repositories for orpo:

Users that are interested in orpo are comparing it to the libraries listed below

• Asap7772 / understanding-rlhf
  Learning from preferences is a common paradigm for fine-tuning language models. Yet, many algorithmic design decisions come into play. Ou…
  ☆28Updated last year
• abaheti95 / LoL-RL
  Advantage Leftover Lunch Reinforcement Learning (A-LoL RL): Improving Language Models with Advantage-based Offline Policy Gradients
  ☆26Updated 7 months ago
• upiterbarg / lintseq
  [ICLR 2025] "Training LMs on Synthetic Edit Sequences Improves Code Synthesis" (Piterbarg, Pinto, Fergus)
  ☆19Updated 2 months ago
• liziniu / policy_optimization
  Code for Paper (Policy Optimization in RLHF: The Impact of Out-of-preference Data)
  ☆28Updated last year
• formll / resolving-scaling-law-discrepancies
  ☆18Updated 9 months ago
• rosieyzh / openrlhf-pretrain
  Code for "Echo Chamber: RL Post-training Amplifies Behaviors Learned in Pretraining"
  ☆11Updated last week
• facebookresearch / dualformer
  implementation of dualformer
  ☆15Updated last month
• allenai / sso
  Repository for Skill Set Optimization
  ☆12Updated 9 months ago
• dunzeng / MORE
  Code for EMNLP'24 paper - On Diversified Preferences of Large Language Model Alignment
  ☆16Updated 8 months ago
• janphilippfranken / sami
  Self-Supervised Alignment with Mutual Information
  ☆17Updated 11 months ago
• tml-epfl / icl-alignment
  Is In-Context Learning Sufficient for Instruction Following in LLMs? [ICLR 2025]
  ☆29Updated 3 months ago
• dangxingyu / rnn-icrag
  Official repository of paper "RNNs Are Not Transformers (Yet): The Key Bottleneck on In-context Retrieval"
  ☆26Updated last year
• Linear95 / DSP
  Domain-specific preference (DSP) data and customized RM fine-tuning.
  ☆25Updated last year
• weigq / openview_quicklook
  ☆33Updated last month
• katiekang1998 / reasoning_generalization
  ☆31Updated 3 months ago
• qcznlp / uncertainty_attack
  ☆16Updated 7 months ago
• cmu-l3 / neurips2024-inference-tutorial-code
  NeurIPS 2024 tutorial on LLM Inference
  ☆42Updated 4 months ago
• holarissun / RewardModelingBeyondBradleyTerry
  official implementation of ICLR'2025 paper: Rethinking Bradley-Terry Models in Preference-based Reward Modeling: Foundations, Theory, and…
  ☆54Updated 3 weeks ago
• ethz-spylab / superhuman-ai-consistency
  ☆29Updated last year
• yidingjiang / ado
  The repository contains code for Adaptive Data Optimization
  ☆23Updated 4 months ago
• alexrame / rewardedsoups
  Rewarded soups official implementation
  ☆58Updated last year
• sail-sg / dice
  Official implementation of Bootstrapping Language Models via DPO Implicit Rewards
  ☆43Updated last week
• ctlllll / reward_collapse
  ☆27Updated last year
• UW-Madison-Lee-Lab / Expressive_Power_of_LoRA
  Code for "The Expressive Power of Low-Rank Adaptation".
  ☆20Updated last year
• jiahai-feng / binding-iclr
  ☆13Updated last year
• yikangshen / megablocks
  ☆20Updated 10 months ago
• felixzli / synthetic_pretraining
  ☆39Updated 2 years ago
• IBM / ColPret
  Efficient Scaling laws and collaborative pretraining.
  ☆16Updated 2 months ago
• limenlp / safer-instruct
  This is the oficial repository for "Safer-Instruct: Aligning Language Models with Automated Preference Data"
  ☆17Updated last year
• Shentao-YANG / Preference_Grounded_Guidance
  Source codes for "Preference-grounded Token-level Guidance for Language Model Fine-tuning" (NeurIPS 2023).
  ☆16Updated 3 months ago