Alternatives and similar repositories for orpo

Users that are interested in orpo are comparing it to the libraries listed below

• tianjunz / TEMPERA
  ☆46Updated 2 years ago
• alexrame / rewardedsoups
  Rewarded soups official implementation
  ☆62Updated 2 years ago
• janphilippfranken / sami
  Self-Supervised Alignment with Mutual Information
  ☆20Updated last year
• sail-sg / dice
  Official implementation of Bootstrapping Language Models via DPO Implicit Rewards
  ☆47Updated 9 months ago
• liziniu / policy_optimization
  Code for Paper (Policy Optimization in RLHF: The Impact of Out-of-preference Data)
  ☆28Updated 2 years ago
• RLHFlow / Directional-Preference-Alignment
  Directional Preference Alignment
  ☆58Updated last year
• deeplearning-wisc / args
  ☆46Updated last year
• facebookresearch / rlfh-gen-div
  This is code for most of the experiments in the paper Understanding the Effects of RLHF on LLM Generalisation and Diversity
  ☆47Updated 2 years ago
• Jiacheng-Zhu-AIML / AsymmetryLoRA
  Preprint: Asymmetry in Low-Rank Adapters of Foundation Models
  ☆37Updated last year
• Asap7772 / understanding-rlhf
  Learning from preferences is a common paradigm for fine-tuning language models. Yet, many algorithmic design decisions come into play. Ou…
  ☆32Updated last year
• Edward-Sun / easy-to-hard
  Easy-to-Hard Generalization: Scalable Alignment Beyond Human Supervision
  ☆124Updated last year
• ryoungj / BoLT
  Code for "Reasoning to Learn from Latent Thoughts"
  ☆124Updated 9 months ago
• liziniu / GEM
  Code for Paper (Preserving Diversity in Supervised Fine-tuning of Large Language Models)
  ☆51Updated 8 months ago
• Linear95 / DSP
  Domain-specific preference (DSP) data and customized RM fine-tuning.
  ☆25Updated last year
• Shentao-YANG / Preference_Grounded_Guidance
  Source codes for "Preference-grounded Token-level Guidance for Language Model Fine-tuning" (NeurIPS 2023).
  ☆17Updated last year
• gregorbachmann / Next-Token-Failures
  ☆107Updated last year
• rosieyzh / openrlhf-pretrain
  Code for "Echo Chamber: RL Post-training Amplifies Behaviors Learned in Pretraining"
  ☆26Updated 3 months ago
• Pranjal2041 / AdaptiveConsistency
  Let's Sample Step by Step: Adaptive-Consistency for Efficient Reasoning with LLMs
  ☆40Updated last year
• ctlllll / reward_collapse
  ☆26Updated 2 years ago
• ruizheng20 / gpo
  The code of paper "Toward Optimal LLM Alignments Using Two-Player Games".
  ☆17Updated last year
• holarissun / RewardModelingBeyondBradleyTerry
  official implementation of ICLR'2025 paper: Rethinking Bradley-Terry Models in Preference-based Reward Modeling: Foundations, Theory, and…
  ☆70Updated 9 months ago
• safety-research / SHADE-Arena
  ☆22Updated 7 months ago
• princeton-pli / what-makes-good-rm
  [NeurIPS 2025] What Makes a Reward Model a Good Teacher? An Optimization Perspective
  ☆42Updated 4 months ago
• haozheji / exact-optimization
  ICML 2024 - Official Repository for EXO: Towards Efficient Exact Optimization of Language Model Alignment
  ☆57Updated last year
• formll / resolving-scaling-law-discrepancies
  ☆20Updated 2 months ago
• jiahai-feng / binding-iclr
  ☆16Updated last year
• sylinrl / CalibratedMath
  Teaching Models to Express Their Uncertainty in Words
  ☆39Updated 3 years ago
• shizhediao / Black-Box-Prompt-Learning
  Source code for the TMLR paper "Black-Box Prompt Learning for Pre-trained Language Models"
  ☆57Updated 2 years ago
• socialfoundations / tttlm
  Test-time-training on nearest neighbors for large language models
  ☆49Updated last year
• princeton-nlp / unintentional-unalignment
  [ICLR 2025] Unintentional Unalignment: Likelihood Displacement in Direct Preference Optimization
  ☆32Updated 2 weeks ago