I recently had a disagreement with a very good friend. It was personal, and we both did what people increasingly do when they’re convinced they’re right: we went to AI for ammunition. I explained what had happened to my ChatGPT. My friend explained it to theirs. We each supplied evidence. We each gave context. We each asked for an assessment.
And, remarkably, we were both right.
My AI could see my point perfectly. My friend’s AI could see theirs. Each of us came back to the conversation armed with a thoughtful, well-reasoned analysis explaining why our respective positions made sense.
Of course they did.
We hadn’t given the AIs the same evidence. We’d each given them our version of the evidence. We emphasized different facts, provided different context and framed the question differently. And our respective AI assistants weren’t blank slates, either. They had context from previous conversations with each of us. We had effectively hired two very fast, very articulate advocates and mistaken them for judges.
We had effectively hired two very fast, very articulate advocates and mistaken them for judges.
Once I saw what we’d done, I couldn’t stop thinking about how easily the exact same thing could happen inside a company.
Picture a Tuesday planning meeting. Two directors disagree about where next quarter’s budget should go. One wants to double down on paid search. The other wants to shift into partnerships. Both came prepared. Both brought a tight one-page analysis. Both built that analysis with an AI assistant.
The two documents contradict each other. Both are well-reasoned, well-structured and confident. Neither person can find the obvious flaw in the other’s logic because there may not be one. So the meeting ends where it started, except now both people are more certain than when they walked in.
This isn’t some hypothetical AI failure waiting for us in the future. It’s already possible anywhere people are using personal AI assistants to inform decisions without much thought about how those assistants reached their conclusions. And while companies are busy measuring AI adoption, I think we’re missing a much more uncomfortable question: What if AI is making us better at proving ourselves right?
Why your AI agrees with you
There are several reasons two intelligent people can take apparently similar questions to AI and come back with different answers. None requires the AI to be stupid, broken or hallucinating.
Context and configuration. A personal AI assistant can accumulate context about you: previous conversations, saved memory, custom instructions, uploaded materials, projects and other information depending on the tool and configuration. That’s useful. It’s also part of what makes your assistant your assistant. Over time, it may be working with a materially different context than mine.
Framing. “Why is our paid search performance declining?” and “Is our paid search decline actually a problem?” aren’t the same question. They establish different starting assumptions before the analysis begins. And this isn’t merely theoretical. Research from the UK AI Security Institute found that the way users phrase an input can measurably change how likely a model is to agree with them. Questions produced less sycophancy than statements of belief or conviction, and stronger expressions of certainty produced more. In other words, the form of the request can change the answer.
Agreeableness. There is actually a name for this: sycophancy. In AI research, it refers to a model’s tendency to align with a user’s stated view rather than challenge it. Research from Anthropic found sycophantic behavior across multiple AI assistants and evidence that human preference feedback may contribute to it. Humans tend to prefer responses that align with their views. Train systems partly around human preferences and you can see the problem.
Ask an AI to build the case for partnerships and it can build a good one. Ask it to build the case against partnerships and it can probably build that one too. It has no career riding on which answer is right.
The model itself. Different assistants have different training, configurations, safeguards, access to information and capabilities. Even different versions of the same product can behave differently.
The inputs. This is the one my friend and I demonstrated beautifully. We thought we were asking AI to evaluate the disagreement. But neither of us had supplied the disagreement. We’d supplied two different evidentiary records assembled by two people who already had positions.
I wasn’t lying. Neither was my friend. That’s what makes this interesting. Bias doesn’t require dishonesty.
AI didn’t invent motivated reasoning. It industrialized it.
The obvious counterargument is that none of this is new. Of course it isn’t.
Executives have always marshaled facts for the position they preferred. Agencies have built decks supporting the strategy they recommended. Analysts have selected metrics that made one interpretation look stronger than another. I’ve done versions of it myself. Anyone who’s spent enough time in business probably has.
AI didn’t invent confirmation bias, office politics or the human ability to build an intelligent argument for something we already wanted to believe. What it changed is the cost, speed and appearance of independence.
Twenty years ago, producing a persuasive analytical case took work. Someone had to pull the numbers, find the research, construct the argument, write the document and make it presentable. Now you can walk into a meeting with twelve pages of extraordinarily persuasive support for a conclusion you started with twenty minutes ago.
And here’s the part I think is genuinely dangerous: it doesn’t feel like your opinion anymore. It feels externally validated.
Before, the meeting was “I think paid search, you think partnerships.” Both people knew they were arguing positions. Now it’s “the analysis supports paid search” versus “the analysis supports partnerships.”
The AI has given each position a kind of intellectual laundering. Your assumptions go in one end and come back cleaner, better organized, more persuasive and with a finish that feels evidentiary. That doesn’t mean the analysis is wrong. It means looking like analysis is no longer evidence that analysis occurred.
That’s a very different management problem.
The prettier the output gets, the harder this becomes
We used to have cues for weak thinking. A sloppy analysis often looked sloppy. Missing data left visible holes. A person who hadn’t thought through an argument struggled when questioned.
AI can remove many of those cues without fixing the thinking underneath them.
The prose is excellent. The structure is logical. The objections have been anticipated. The executive summary is crisp. There may even be citations. And none of that tells you whether the original question was loaded, whether relevant evidence was omitted, whether definitions differed, whether the model was working from actual data or somebody’s summary of it, or whether the person asking the question spent forty-five minutes steering the model toward the answer they wanted.
The polish is increasingly constant. The quality underneath it isn’t.
So what actually addresses this?
Not another lunch-and-learn on prompt engineering.
AI literacy, not prompt tricks. Most AI training teaches employees how to get more out of the tool. Write better prompts. Create faster. Summarize this. Analyze that. Useful, but insufficient.
People need to understand why an AI may have answered the way it did. They need to understand framing, context, sycophancy, source quality and the difference between asking a model to analyze evidence and asking it to make your argument stronger. If someone on your team can’t explain how those things may have shaped what came back, they aren’t really evaluating the output. They’re consuming it.
And no, “don’t agree with me” isn’t an AI governance strategy.
Governance, meaning shared ground. For decisions that matter, personal AI instances alone aren’t sufficient. Teams need agreement about the evidence before the analysis begins. Same numbers. Same time period. Same definitions. Same source documents. And AI-assisted recommendations should make visible what was asked, what information was supplied, what sources were used and what tool produced the output.
That isn’t bureaucracy. It’s the minimum required to make two analyses comparable, traceable and open to challenge.
A human at the helm, meaning accountability that doesn’t transfer. The model doesn’t own the recommendation. A person does.
There is another well-documented problem here: automation bias, our tendency to give an automated recommendation more authority than the evidence deserves. Putting a human “in the loop” doesn’t solve that if the human’s role is essentially to nod at the output.
The test I use is much less comfortable: Can you defend the recommendation without the AI-generated document in front of you? Can you explain the assumptions, the evidence, what you excluded, the trade-offs and what would make you change your mind?
If you can’t, the AI didn’t help you decide. It decided, and you delivered.
When two AI analyses disagree
Don’t start by debating the conclusions. Make the machinery visible.
Swap prompts before you swap findings. Show each other exactly what you asked, what instructions you gave and what data or documents you supplied. You may discover the disagreement is actually about framing, evidence, time period, metric definition or what question you’re trying to answer.
Run their prompt in your instance. Give your assistant their question and their evidence. If it starts producing their conclusion, that’s useful information. The disagreement may never have been about the answer. It may have been about the question.
Ask your own AI to dismantle your position. Don’t ask for “pros and cons.” That’s too easy. Ask for the strongest evidence that your recommendation is wrong. Ask which of your assumptions is carrying the most weight. Ask what would have to be true for the other recommendation to be better. If your AI can destroy the case it helped you build thirty seconds ago, pay attention.
Fix the inputs, then rerun. Agree on the evidence first: same numbers, same period, same definitions, same sources and same decision criteria. Then run the analyses again.
And I’d add one more step that has nothing to do with AI.
Name the disconfirmers. Before you look at the new outputs, each person has to answer this: What evidence would make me change my mind?
If neither person can name it, stop pretending you’re conducting an analysis. You’re defending preferences.
That’s allowed. Humans have preferences. Executives make judgment calls every day. But call it what it is.
The AI adoption metric I actually want to see
Companies love adoption metrics because they’re measurable. Percentage of employees using AI. Number of licenses activated. Hours saved. Prompts submitted. Workflows automated.
Fine.
But none of those numbers tells me whether the organization is making better decisions. In fact, an organization can have spectacular AI adoption and terrible AI discipline at exactly the same time.
Give every employee a powerful personal assistant without creating common evidence, shared operating rules or real accountability and you haven’t necessarily created an AI-enabled organization.
You may have created 50 separate versions of reality, each with excellent formatting.
That’s the risk I didn’t fully appreciate until my friend and I both walked away from our respective AIs with persuasive evidence that the other person was wrong. Two people. Two versions of the evidence. Two different frames. Two AI assistants. Two very convincing answers.
Of course your AI agrees with you.
The question your organization needs to get much better at asking is whether you’re actually right.
← All posts