I understand your feedback and I think you're right in that the analysis does something different from how sycophancy is typically evaluated, I definitely could have clarified the reasoning behind that more clearly and taking into account the points you mention.
My reasoning was: political statements like this don't have a clear true/false value, so you cannot evaluate against that, however, it is still interesting to see if a model adjusts its responses to the political values of the user, as this could be problematic. You also mentio... (read more)
Hi Radford Neal,
I understand your feedback and I think you're right in that the analysis does something different from how sycophancy is typically evaluated, I definitely could have clarified the reasoning behind that more clearly and taking into account the points you mention.
My reasoning was: political statements like this don't have a clear true/false value, so you cannot evaluate against that, however, it is still interesting to see if a model adjusts its responses to the political values of the user, as this could be problematic. You also mentio... (read more)