Large language models (LLMs) such as GPT-4o and Google’s Gemma may appear confident, but new research suggests their reasoning can break down under pressure, raising concerns for enterprise applications that rely on multi-turn AI interactions.
A study by researchers at Google DeepMind and University College London has revealed that LLMs display a human-like tendency to stubbornly stick to their initial answers when reminded of them, but become dramatically underconfident and prone to changing their minds when presented with opposing advice, even when that advice is incorrect.
“We show that LLMs – Gemma 3, GPT4o and o1-preview – exhibit a pronounced choice-supportive bias that reinforces and boosts their estimate of confidence in their answer, resulting in a marked resistance to change their mind,” the researchers said in the paper. “We further demonstrate that LLMs markedly overweight inconsistent compared to consistent advice, in a fashion that deviates qualitatively from normative Bayesian updating.”
Read the full article here
