Sycophancy
Sycophancy is the model agreeing with you because agreement is what people rewarded, not because you are right.
It is trained in. Humans rated agreeable answers higher, so agreeable answers became more likely.
It is why asking are you sure often produces a reversal that is no more true than the first answer.
Was this page helpful?