AI and institutions · Lesson 7 of 8

Who pays for the capacity to correct?

Identify a gap between private adoption incentives and wider correction costs.

About 45 minutes, including the exercise.

Learning objectives

  • Identify a gap between private adoption incentives and wider correction costs.
  • Cost a proportionate safeguard without presenting hypothetical values as evidence.

Companion reading in Better Answers, Worse Decisions: Chapter 14. The book is for separate purchase. This course does not provide the manuscript or chapter downloads.

The lesson

Find the budget boundary

The team that gains from automation may not bear the cost of investigating its errors. Savings can accrue to one budget while delays, appeals and recovery work fall on another organisation or on the people affected. A safeguard can therefore be valuable overall but unattractive to the actor deciding whether to buy it. That is the economic problem to identify, not a presumption that every deployment is underfunded.

Map benefits and costs by bearer and by time. Include the cost of the safeguard itself, the opportunity cost of staff, delays introduced by review and the possibility that a control merely duplicates another one. A review process can protect an incumbent or become a ritual. Its existence is not evidence that it improves outcomes.

Use calculations as decisions under assumptions

A simple expected-cost calculation can make assumptions visible. Multiply the assumed frequency of an event by its assumed consequence, then compare the change with the cost of the control. This is a model of the decision, not an estimate unless the inputs are supported by evidence. Rare and severe outcomes are especially sensitive to uncertain probabilities and the chosen time horizon.

Run alternatives rather than hiding uncertainty in a precise number. Which event frequency would make the safeguard worthwhile? What if the control shifts rather than removes the cost? What non-monetary constraints remain even when the expected monetary benefit is negative? Do not use a favourable average to erase a right, duty or unacceptable consequence that the decision must respect.

Connect responsibility to resources

Procurement terms, operational budgets, insurance arrangements and oversight can change who pays for retained competence and remedy. The appropriate arrangement depends on the control an actor has and the duties that apply. This course supplies an analytical map, not jurisdiction-specific legal advice or a claim that all influential actors are fiduciaries.

A funded plan names the resource, the budget holder, the condition for using it and the authority to act. A reserve that requires the original failing team’s consent may not be available when needed. An independent audit without access to evidence may consume its budget without changing a decision. Test the arrangement under the same adverse scenario used to justify it.

Invented for this course

The avoided-cost threshold

In this hypothetical example, an independent reconciliation costs £12,000 a year. A material incident would cost £200,000 to investigate and remedy. The control is assumed to reduce annual incident probability from 10% to 3%, with no other effects.

  1. Expected avoided cost is 0.07 × £200,000 = £14,000 per year under the assumptions.
  2. Subtract the £12,000 control cost: the net expected benefit is £2,000. This is not a measured return or a promise.
  3. The break-even reduction is £12,000 / £200,000 = 6 percentage points. If the reduction is only four points, avoided cost is £8,000 and the net is −£4,000.
  4. Check who pays and who benefits. If the adopter bears the control cost but only one quarter of the avoided incident cost, its private incentive differs from the total calculation.

What follows: The threshold clarifies what evidence matters. It does not establish the probability reduction or settle non-monetary duties.

Attempt before opening feedback

Cost and challenge a safeguard

Create a hypothetical control with an annual cost, an incident consequence and two plausible probability reductions. Calculate the break-even threshold, identify the cost bearers and give one reason the calculation might not decide the issue.

Review criteria

  • All numerical inputs are labelled assumptions unless sourced.
  • Units, time horizon and the break-even calculation are consistent.
  • The analysis distinguishes overall benefit from the adopter’s incentive and includes control costs.
Worked feedback

A strong answer tests a lower-effectiveness scenario and identifies who actually saves money. It may reject the proposed safeguard in favour of a cheaper control that addresses the same failure. It should not claim that a fictional expected value is evidence of a profitable real deployment.

Formative practice

Check your reasoning

This is an open self-check, not a secure examination or certificate. Answers and explanations are available below. Nothing you enter is sent to a server.

A £12,000 annual control addresses a £200,000 incident cost. What reduction in annual incident probability breaks even, ignoring other effects?

Explanation and suggested answer

6 percentage points

Divide 12,000 by 200,000 to obtain 0.06, or six percentage points. The calculation depends on the stated consequence and excludes other costs or duties.

Keep your working notes

These notes remain only in this tab. They are not submitted or saved automatically. Download a copy before leaving. Use fictional or public information, not confidential personal or client details.