Distinguish different outputs from independent evidence.
About 45 minutes, including the exercise.
Learning objectives
Distinguish different outputs from independent evidence.
Explain why a shared failure can defeat apparent corroboration.
Companion reading in Better Answers, Worse Decisions: Chapters 2, 3 and 10. The book is for separate purchase. This course does not provide the manuscript or chapter downloads.
The lesson
Trace the route to the conclusion
Three polished reports may have one evidential ancestor. They can use different wording, interfaces and authors while relying on the same model, retrieval source or copied assumption. Counting reports then overstates the amount of corroboration. Start with a dependency map: source observation, transformation, model, recommendation and final decision. Mark where one branch inherits another branch’s result.
Independence is not a demand that every person work alone. Shared standards can reduce arbitrary inconsistency, and shared tools can make expensive expertise available. The question is whether a second path can reveal a material error that the first path is likely to miss. An alternative brand is not enough; nor is an unaided human judgement automatically independent or better.
A small probability example
Suppose, only for illustration, that each of two checks has a 10% probability of missing a particular defect. If their misses are independent, both miss it with probability 1%. If they always miss exactly the same defects, both miss with probability 10%. The marginal rate is identical but the combined protection is not. These are assumptions for a lesson, not measured rates for any product.
Real dependencies sit between simple extremes and can change by failure type. Two checks may disagree on arithmetic while sharing a mistaken definition of the task. Record the kinds of errors each check is designed to expose. A numerical independence assumption should come from evidence or be examined through sensitivity analysis, not inferred from the number of vendors.
Preserve disagreement before resolving it
One inexpensive design change is temporal separation. Ask a reviewer to record an initial assessment before seeing the common recommendation, then compare the reasons. This does not prove that the initial assessment is correct. It preserves information that could otherwise disappear through anchoring or conformity.
Keep the original evidence and the reason for changing the decision. A disagreement log is useful only if someone can investigate an important difference and act on it. Do not reward disagreement for its own sake: a reviewer who introduces random variation has not supplied a useful independent path. Test whether the alternative detects errors, at what cost, and whether the organisation can adjudicate the conflict.
Invented for this course
Two suppliers, one source
A fictional procurement team receives two risk reports. One uses a commercial assistant and the other a different interface, but both cite the same outdated spreadsheet. The team treats their agreement as two independent confirmations.
Identify the common spreadsheet as the evidential dependency. Different prose and vendor names do not remove it.
Check the relevant figures against a separately maintained transaction record and its update date.
Preserve both original reports and record whether the independent record changes the decision. Correct the shared source as well as the individual answer.
What follows: The valuable third check is a different evidential route, not another version of the same output.
Attempt before opening feedback
Draw a dependence map
Map three recommendations about a fictional decision. Give each its sources, model or method, and exposure to previous recommendations. Design one check that can fail differently.
Review criteria
Shared sources and copied assumptions are visible.
The proposed alternative targets a specified failure rather than generic diversity.
You identify the cost of the check and how disagreement reaches a decision.
Worked feedback
A strong map might reveal that two analysts and a dashboard all inherit the same data extraction. A sample reconciled directly against original records can test that extraction. It will not establish that the decision rule itself is justified, so a separate examination of the rule may still be needed.
Formative practice
Check your reasoning
This is an open self-check, not a secure examination or certificate. Answers and explanations are available below. Nothing you enter is sent to a server.
Keep your working notes
These notes remain only in this tab. They are not submitted or saved automatically. Download a copy before leaving. Use fictional or public information, not confidential personal or client details.