Don’t Take AI’s Word for It

This is a conversation about evidence, bias, consensus—and the danger of confusing confidence with truth

ASK MISTER ROBOT is Traveling Boy’s Q & A about a variety of topics that keep people awake at night. The answers come from Mr. Robot, an AI model, who scours the internet for answers. Don’t take his answers like gospel truth, rather consider his answers as an educated opinion. We welcome your questions and invite you to join in the conversation.

What if the most dangerous thing about artificial intelligence isn’t that it can get things wrong, but that it can sound so convincing when it does?

That question emerged from a simple conversation with Mr. Robot.

I began by asking whether the Clinton Foundation still exists. The answer was straightforward. Then I asked whether Chelsea Clinton received $8 million for her wedding. Again, I received a confident answer.

But then I asked about something more controversial: Was Barack Obama’s birth certificate posted on the White House website a fake?

Mr. Robot gave me the conventional answer: no, Hawaii had authenticated the underlying record.

But I wasn’t asking only what the authorities said. I pointed out something I had personally observed in the document: the PDF contained multiple digital layers.

Mr Robot initially explained the discrepancy away. When I challenged the reasoning, however, it acknowledged something important: the layers were real. What remained disputed was what those layers actually demonstrated.

That distinction changed the conversation.

The Problem Isn’t Always the Facts

This is where the issue of bias becomes much more interesting.

An AI can truthfully say, “I base my answer on facts, evidence and science.” But where do those facts come from? What sources are represented? Which sources receive greater weight? Which interpretations are considered credible? And what happens when the prevailing consensus is wrong?

History gives us plenty of reasons to be cautious.

Imagine asking an intelligent machine centuries ago whether the Earth moves around the Sun. If that machine had access primarily to the prevailing authorities of its time, it might have concluded that Galileo was mistaken.

Or imagine asking it about evolution during Darwin’s era. The answer would have reflected the scientific and cultural understanding available at that particular moment.

The lesson isn’t that experts are useless or that consensus is always wrong.

The lesson is more subtle:

Consensus is evidence. It is not proof.

Galileo Wasn’t Wrong Because He Was Outnumbered

Scientific progress often begins with someone challenging an assumption that everyone else has become comfortable accepting.

That doesn’t mean every dissenter is a Galileo. Most unconventional ideas are wrong. Conspiracy theories aren’t automatically true simply because they’re unpopular.

But neither should an idea be dismissed merely because it conflicts with the prevailing narrative.

The real question should be:

What does the evidence actually show?

And just as importantly:

What evidence would prove us wrong?

Those two questions are surprisingly powerful.

The Clinton Test

The same principle applies to politics.

Consider Bill Clinton’s famous statement that he had experimented with marijuana but “didn’t inhale.”

A careful examination separates what is known from what is merely claimed. Clinton admitted using marijuana. He later said that he had tried to inhale but couldn’t.

We can establish that he used marijuana because he said so himself. But whether he actually inhaled is something the public record cannot independently establish with certainty.

That’s a small example, but it illustrates an important principle:

A person’s statement is evidence. It isn’t automatically the truth simply because the person said it.

And that standard should apply equally to politicians we like and politicians we dislike.

The Same Standard Must Apply to Trump

The Russia controversy presents an even more complicated example.

The Steele dossier contained serious allegations about Donald Trump and Russia. But saying that the dossier existed is not the same as saying that its allegations were proven.

At the same time, rejecting the dossier doesn’t mean nothing happened. Russian interference in the 2016 election was established independently of the dossier, and there were documented contacts between people associated with the Trump campaign and Russian-linked individuals.

Again, the responsible position isn’t necessarily the most politically convenient one.

It’s the one that distinguishes:

What is established.
What is strongly supported.
What is disputed.
What is merely alleged.
And what is speculation.

The AI Problem

This is where artificial intelligence presents a unique challenge.

An AI doesn’t have to be deliberately programmed to favor one political ideology to produce biased answers.
It can inherit biases from its training material.
It can reflect the assumptions of the institutions that produce information.
It can learn that certain claims are commonly associated with “misinformation” and become overly skeptical of them.
It can give greater weight to sources that are considered authoritative while giving insufficient consideration to dissenting evidence.
And perhaps most importantly, it can confuse consensus with truth.

The result is particularly dangerous because AI doesn’t usually sound uncertain.

It can produce a polished, articulate explanation that gives the impression that the matter is settled—even when the underlying evidence is considerably more complicated.