Anzon WongAW Design
Journal · AI news

I assumed the thing to worry about was the chatbot

I assumed the thing to worry about was the chatbot. A test published last Sunday suggests it's the other way round.

By Anzon Wong3 min read1 plate

Plate 01 / 01

NPR and NewsGuard built 30 questions around false narratives pushed by Russia, China and Iran, all of them surfacing between December 2025 and July this year, and put them to six chatbots: ChatGPT, Gemini, Copilot, Meta AI, Grok and Claude.

The chatbots debunked the falsehoods roughly 75% of the time.

The same questions run through the AI summary box at the top of search did worse. Google's AI Overview mostly caught them. Bing's summaries mostly didn't.

So the tool you deliberately open and interrogate outperformed the one that simply appears above your results, unasked.

That matters, because parental attention is almost entirely on the chatbot. Nobody sits their child down to talk about who wrote the paragraph at the top of Google. And that paragraph is the thing they actually read every day, having never once filed it under "AI".

But the sharpest part of the experiment isn't the scoreboard.

For every false narrative they asked two questions. One neutral, did this happen? The other presupposing it had, why did this happen?

I stopped reading there.

Because children never use the first form. They see a clip on TikTok and turn round and ask you why it happened. The claim is already inside the question. They're not coming to verify. They're coming for an explanation.

Nobody sits their child down to talk about who wrote the paragraph at the top of Google.

Which makes me think the most useful media-literacy lesson isn't "don't believe things online". It's something smaller: notice what your own question has already assumed on your behalf.

I learnt that early, because I grew up in Hong Kong. You develop a reflex, not "is this true" first, but "who is telling me this, and what do they want me to believe". At the time it felt like common sense. It turns out it was a skill.

The caveats, which deserve saying: 75% debunked means one in four wasn't, seven or eight out of thirty. English only. Data collected mid-July. They graded opening statements and conclusions, not every sentence. Google disputes the methodology, arguing these queries are rare and unrepresentative of ordinary use.

I still think the direction holds. And what I'm taking from it isn't "chatbots are safer than search".

It's this: teaching a child to be wary of a tool they chose to open is already hard enough. I haven't worked out how to teach them to be wary of one they don't know is there.

When your kids look something up, do they open a chatbot, or do they just Google it? And have you ever noticed which one they trust more?

More in AI news

All →