New experiments by Valerio Capraro show that incorrect AI advice can make people less accurate but more confident. When an AI was available, participants stopped declining answers (from 36%–44% down to 3%–6%) and overall accuracy fell from 27.5% to 9.2%. The effect persisted even when AI prompts appeared automatically, and financial incentives only partially reduced it. Capraro warns AI should support — not supplant — human judgment, and that children may miss out on important learning by forgoing doubt.
Study: Wrong AI Advice Makes People Less Accurate — But Far More Confident
Representative Image (AI-generated)
New research from Valerio Capraro, a psychology professor at the University of Milan‑Bicocca, shows that consulting AI can turn healthy doubt into misplaced confidence. In a controlled set of experiments, participants answered obscure movie questions while an AI system—deliberately chosen because it consistently gave wrong answers—offered suggestions. The results highlight a surprising and potentially worrying effect of AI assistance.
What the Experiments Tested
Researchers asked participants six obscure questions about movie details. The team intentionally selected items the AI model repeatedly answered incorrectly so they could isolate how people respond to AI advice independent of its accuracy. Participants could choose to answer or decline; in some trials, financial rewards and penalties were used to incentivize accuracy.
Key Findings
When working alone, participants declined to answer roughly 36%–44% of the time across two experiments. With AI advice available, refusal rates plunged to just 3%–6%. Accuracy also fell: participants were correct 27.5% of the time alone but only 9.2% when AI input was present.
Sensor Tower
Capraro concludes that AI appears to lower the internal confidence threshold people use before committing to an answer, effectively making them more willing to answer even when they are wrong. Remarkably, the effect persisted even when AI suggestions appeared automatically without being requested, indicating it is not limited to situations where users actively seek help.
Incentives, Kids, and Broader Concerns
Introducing monetary rewards and penalties made participants somewhat more cautious and modestly improved accuracy, but did not eliminate the pattern of reduced withholding and lower correctness when AI was present. Capraro stresses that AI should augment human judgment rather than replace it.
He also warns this dynamic could affect children, who may miss learning opportunities if instant AI answers reduce their chances to experience productive doubt. The study cites anecdotal reports that younger users sometimes consult chatbots even during face‑to‑face conversations. As Capraro notes, doubt is not a failure — it is often where real understanding begins.
Image and Sources: The article included an AI‑generated representative image. Ancillary visuals referenced Sensor Tower. The research was reported via IBM Think and led by Valerio Capraro at the University of Milan‑Bicocca.
Help us improve.
