Most AI users say they should verify chatbot answers, yet many do so only sometimes or less often, according to a survey by cybersecurity company F-Secure.
The survey of 1,500 AI users in the U.S. and U.K. found that nearly 90% said they were responsible for verifying AI answers, yet 70% said they checked them only sometimes, rarely, or never.
“Consumers are busy and have complex, messy lives,” explained F-Secure Vice President of Research Laura James.
“They make tradeoffs every day around what digital tools to use and for what,” she told TechNewsWorld. “AI is no different.”
AI can be very helpful, but it has its challenges, such as hallucinations or low-quality answers, she continued. “Consumers use it knowing these issues exist, as a pragmatic choice, prioritizing getting things done,” she said.
“They know that the answers cannot always be trusted,” she noted, “but the alternative for them in the moment is worse.”
“It can be quite hard to check things out,” she maintained, “as search engines don’t always work well and other information sources are paywalled.”
She added that consumers are turning to chatbots with questions they cannot get answered in other ways, such as medical advice that may be too expensive or slow to access.
‘Just Give Me the Answer’
One reason AI users skip vetting answers is it defeats part of the reason people use AI in the first place: speed, asserted Eric Turney, president of The Monterey Company, a custom branded merchandise company in Bend, Ore.
“If AI gives you an answer in 15 seconds but properly checking it takes another 10 minutes, people naturally start taking shortcuts,” he told TechNewsWorld.
The gap between “talking the talk” and “walking the walk” discovered by F-Secure’s researchers isn’t limited to the use of AI. “For example, there is almost always a gap between the people who claim ‘they should eat healthier’ and their food choices,” observed Juan Londoño, a policy analyst with the Cato Institute, a Washington, D.C. think tank.
The gap in the F-Secure survey can happen for various reasons, but one of them is the effort involved in fact-checking, he continued.
“Sometimes people are looking for a quick answer, and they do not think they should invest the time to confirm,” he told TechNewsWorld.
“When an individual is not very knowledgeable about a certain topic, it can be hard to identify what has to be fact-checked,” he said. “We saw this trend on social media, where people did not even realize they had to question the content they were presented with.”
“I see that temptation myself,” he admitted. “The more useful AI becomes, the easier it is to move from ‘help me research this’ to ‘just give me the answer.'”
It Sounded Right
Greg Sterling, co-founder of Near Media, a market research firm in San Francisco, pointed out that users are becoming more sophisticated about AI. “That may account for some of the reduced fact-checking,” he told TechNewsWorld.
“Laziness or complacency could be another explanation,” he continued, “but it really depends on the situation. No generalization is going to be fully explanatory.”
Nearly half of the AI users surveyed (47.3%) said they stop questioning an answer if it sounds right — more than twice the share who gave any other reason for not verifying a response.
“AI can give a wrong answer in clear, confident language, and that confidence can make the answer feel credible,” noted Mark N. Vena, president and principal analyst at SmartTech Research, a technology advisory firm in Las Vegas.
“If it also matches what you already believe, there’s even less reason to pause and question it,” he told TechNewsWorld.
Tony Xhufi, founder of WEM, an AI-powered product-discovery and price-comparison service based in England, agreed that confident language, a clear explanation and specific details can make an answer feel authoritative, particularly when it agrees with what someone already believes.
“An AI answer can contain those reassuring features even when an important fact is wrong, and fluency can make an unchecked answer feel finished,” he told TechNewsWorld.
“One thing that would help users would be if the chatbot were able to indicate when it was less confident in an answer, rather than producing a response with the same confidence as if it were very sure,” added F-Secure’s James.
Real-Life Decisions, Real-Life Harms
The F-Secure research also found that one in four respondents (25.2%) have followed AI advice on a major real-life decision. That’s proof, the researchers reasoned, that users aren’t just casually using chatbots for casual questions or hypothetical scenarios.
“AI has real beneficial use applications, but they should not replace sound advice from qualified, trained professionals,” cautioned Michelle Lopes Maldonado, associate director of AI policy at the Information Technology & Innovation Foundation, a research and public policy organization in Washington, D.C.
“There are real-life decisions that could have harmful consequences across a spectrum of health, legal, and financial areas,” she told TechNewsWorld.
Relying on AI for medication and dosing decisions, treatment of symptoms that might need urgent care, retirement-account and tax moves, signing or breaking a contract, immigration and court deadlines, or even things that are more mundane such as structural or electrical home repairs — all could result in significant risks or harms if left unverified, she explained.
“The common thread is reversibility,” added Jessica Murphy, CEO and co-founder of True Fit, in Boston, an AI-powered fit and sizing-intelligence company for apparel and footwear.
“If a wrong answer costs you an afternoon, the stakes are limited,” she told TechNewsWorld. “If it could affect a diagnosis, a contract, or a job, that is where a qualified person with accountability needs to remain involved.”
More AI Tasks, Less Verification
According to the survey, using AI for a wider variety of tasks was associated with accepting answers without further verification. Among people who use AI for many different types of tasks, it noted, 59.7% accepted answers without further verification, compared with 28.6% among those who only use AI for low-stakes questions.
“We generally don’t like to edit our work,” noted Rob Enderle, president and principal analyst with the Enderle Group, an advisory services firm in Bend, Ore.
“It just isn’t something most people are trained to do, and even those of us trained to do it often find editing tedious,” he told TechNewsWorld. “So, as we become more comfortable with AI, we naturally reduce our efforts to challenge the AI’s responses.”
“Familiarity breeds trust faster than it should,” added Matt Duren, senior vice president for AI and data at KnowBe4, a security awareness training provider in Clearwater, Fla.
“Every correct answer is a small deposit in the trust account, and after enough deposits, people stop auditing the balance,” he told TechNewsWorld.
“It’s also a workload problem,” he continued. “If you use AI for dozens of tasks a day, verifying each one destroys the productivity gain that made you adopt it, so people rationally triage and then the triage quietly becomes ‘never.'”
“Heavy users are also more likely to be using AI for exactly the higher-stakes tasks where the consequences of an error are greatest, so risk exposure and complacency grow together,” he warned.
Human Behavior Story
Duren argued that the biggest takeaway from the survey is that it’s a human-behavior story, not a technology story.
“We spent the better part of the last two decades teaching people to be skeptical of emails,” he said. “Now we need to build the same reflex for AI output.”
“And it has to be practical,” he continued. “Verify when the stakes are high, not for every question.”
“Organizations should be training employees on this now,” he advised, “because the same person who follows unverified AI advice at home is doing it at work with company data, and increasingly delegating it to an agent that never questions anything.”
“The tools will keep getting better, but ‘sounds correct’ will never be a substitute for ‘actually correct,'” he added.
Read the full article here
