Alternatives and similar repositories for weak-to-strong-preference-optimization

Users that are interested in weak-to-strong-preference-optimization are comparing it to the libraries listed below

• limenlp / verl
  AdaRFT: Efficient Reinforcement Finetuning via Adaptive Curriculum Learning
  ☆41Updated 2 months ago
• horseee / CoT-Valve
  CoT-Valve: Length-Compressible Chain-of-Thought Tuning
  ☆85Updated 6 months ago
• hahahawu / Long-to-Short-via-Model-Merging
  Model merging is a highly efficient approach for long-to-short reasoning.
  ☆80Updated 3 months ago
• NineAbyss / S2R
  This is the official implementation of the paper "S²R: Teaching LLMs to Self-verify and Self-correct via Reinforcement Learning"
  ☆69Updated 4 months ago
• ssmisya / PRMBench
  [ACL' 25] The official code repository for PRMBench: A Fine-grained and Challenging Benchmark for Process-Level Reward Models.
  ☆81Updated 6 months ago
• Zanette-Labs / efficient-reasoning
  ☆66Updated 4 months ago
• SeekingDream / Static-to-Dynamic-LLMEval
  ☆44Updated this week
• rookie-joe / AutoPSV
  ☆49Updated 10 months ago
• OpenBMB / RLPR
  Extrapolating RLVR to General Domains without Verifiers
  ☆151Updated 3 weeks ago
• StarDewXXX / O1-Pruner
  Official repository for paper: O1-Pruner: Length-Harmonizing Fine-Tuning for O1-Like Reasoning Pruning
  ☆87Updated 6 months ago
• hemingkx / TokenSkip
  [EMNLP 2025] TokenSkip: Controllable Chain-of-Thought Compression in LLMs
  ☆177Updated 2 months ago
• sail-sg / CPO
  [NeurIPS 2024] The official implementation of paper: Chain of Preference Optimization: Improving Chain-of-Thought Reasoning in LLMs.
  ☆127Updated 5 months ago
• StarDewXXX / AdaR1
  The official repository of paper "AdaR1: From Long-Cot to Hybrid-CoT via Bi-Level Adaptive Reasoning Optimization"
  ☆18Updated 4 months ago
• sail-sg / sdft
  [ACL 2024] The official codebase for the paper "Self-Distillation Bridges Distribution Gap in Language Model Fine-tuning".
  ☆128Updated 10 months ago
• mlwu22 / RED
  Implementation code for ACL2024：Advancing Parameter Efficiency in Fine-tuning via Representation Editing
  ☆14Updated last year
• PKU-Alignment / aligner
  [NeurIPS 2024 Oral] Aligner: Efficient Alignment by Learning to Correct
  ☆184Updated 7 months ago
• dongguanting / FollowRAG
  The demo, code and data of FollowRAG
  ☆74Updated 2 months ago
• Alsace08 / Chain-of-Embedding
  [ICLR 2025] Code and Data Repo for Paper "Latent Space Chain-of-Embedding Enables Output-free LLM Self-Evaluation"
  ☆78Updated 8 months ago
• AI45Lab / REEF
  The repository of the paper "REEF: Representation Encoding Fingerprints for Large Language Models," aims to protect the IP of open-source…
  ☆62Updated 7 months ago
• THU-KEG / RM-Bench
  [ICLR 25 Oral] RM-Bench: Benchmarking Reward Models of Language Models with Subtlety and Style
  ☆60Updated last month
• hkust-nlp / Laser
  Laser: Learn to Reason Efficiently with Adaptive Length-based Reward Shaping
  ☆52Updated 3 months ago
• UCSB-NLP-Chang / ThinkPrune
  ☆41Updated 4 months ago
• SihengLi99 / LLM-Honesty-Survey
  [2025-TMLR] A Survey on the Honesty of Large Language Models
  ☆59Updated 8 months ago
• yyDing1 / ScaleQuest
  [ACL-25] We introduce ScaleQuest, a scalable, novel and cost-effective data synthesis method to unleash the reasoning capability of LLMs.
  ☆64Updated 10 months ago
• EIT-NLP / Awesome-Latent-CoT
  This repository contains a regularly updated paper list for LLMs-reasoning-in-latent-space.
  ☆154Updated this week
• zjunlp / LightThinker
  [EMNLP 2025] LightThinker: Thinking Step-by-Step Compression
  ☆93Updated 4 months ago
• yafuly / TPO
  Test-time preferenece optimization (ICML 2025).
  ☆162Updated 3 months ago
• ChnQ / MI-Peaks
  ☆50Updated last month
• Joshua-Ren / Learning_dynamics_LLM
  ☆164Updated 3 months ago
• TianHongZXY / RLVR-Decomposed
  Implementation for the paper "The Surprising Effectiveness of Negative Reinforcement in LLM Reasoning"
  ☆100Updated last week