Rafael RafailovDirect Preference Optimization: Your Language Model is Secretly a Reward ModelDisentangling Length from Quality in Direct Preference OptimizationFrom r to Q*: Your Language Model is Secretly a Q-FunctionAll names