Two Agents Disagree
Settles contradictory answers with an experiment instead of a third opinion.
When to reach for it
When two runs, two models, or two sessions tell you opposite things about your own code and both sound equally sure.
What changes
- The disagreement is reduced to a single proposition that one command can settle.
- Each side's evidence is labeled read-from-the-repo, ran-it, or from-memory — and from-memory loses by default.
- You get the specific experiment, and its actual output, as the tiebreaker.
- The losing answer's cause gets named — stale version, wrong file, different question — so the same confusion does not recur.
Pairs with
Two Agents Disagree
Confidence is not evidence, and neither is repetition. Find the artifact that settles it.
1. Reduce it to one proposition
Write the disagreement as a single statement that is either true or false about this repository, right now. "The config is read at build time, not at request time." If it cannot be written that way, the two answers may not actually conflict — go to step 2 first.
2. Check whether they answered the same question
Frequent false conflicts: different file with the same name, different package version, different branch, different environment, one talking about the general case and one about this codebase. Rule these out before treating it as a contradiction.
3. Grade each side's evidence
Label what each answer rests on:
- Ran it — an executed command with output
- Read it — a specific file, at a specific path, in this repo or in the installed package
- From memory — general knowledge, recalled documentation, "usually this works like"
From memory loses to either of the others by default, no matter how detailed it is.
4. Find the tiebreaker artifact
Name the thing in the world that settles it, then go look at it: the installed source under the dependency directory, the lockfile version, a printed value, the actual response, the generated output. Not the documentation for a version you might not have.
5. Run the smallest distinguishing experiment
Design it so the two answers predict different results. Five lines, one log statement, one assertion. Run it. Paste the output.
6. Record the verdict and the cause
State which answer was right, and why the other one was wrong. The reason is the valuable part — a stale version assumption or a misread path will produce the next contradiction too.
Rules
- Do not resolve by asking again and taking the majority. Three guesses do not make a measurement.
- Do not prefer the longer, more confident, or more recent answer. Length is not evidence.
- Do not compromise between them in the implementation. When two answers conflict, the average is wrong in a new way that neither side predicted.
- Do not accept a citation nobody opened. A quoted file path is a claim; the file is the evidence.
- If the experiment cannot distinguish them, say so and design a better one rather than declaring a winner.