AI and institutions · Lesson 7 of 8

Who pays for the capacity to correct?

Identify a gap between private adoption incentives and wider correction costs.

About 45 minutes, including the exercise.

Learning objectives

  • Identify a gap between private adoption incentives and wider correction costs.
  • Cost a proportionate safeguard without presenting hypothetical values as evidence.

Companion reading in Better Answers, Worse Decisions: Chapter 14. The book is for separate purchase. This course does not provide the manuscript or chapter downloads.

The lesson

Find the budget boundary

The team that gains from automation may not bear the cost of investigating its errors. Savings can accrue to one budget while delays, appeals and recovery work fall on another organisation or on the people affected. A safeguard can therefore be valuable overall but unattractive to the actor deciding whether to buy it. That is the economic problem to identify, not a presumption that every deployment is underfunded.

Map benefits and costs by bearer and by time. Include the cost of the safeguard itself, the opportunity cost of staff, delays introduced by review and the possibility that a control merely duplicates another one. A review process can protect an incumbent or become a ritual. Its existence is not evidence that it improves outcomes.

Use calculations as decisions under assumptions

A simple expected-cost calculation can make assumptions visible. Multiply the assumed frequency of an event by its assumed consequence, then compare the change with the cost of the control. This is a model of the decision, not an estimate unless the inputs are supported by evidence. Rare and severe outcomes are especially sensitive to uncertain probabilities and the chosen time horizon.

Run alternatives rather than hiding uncertainty in a precise number. Which event frequency would make the safeguard worthwhile? What if the control shifts rather than removes the cost? What non-monetary constraints remain even when the expected monetary benefit is negative? Do not use a favourable average to erase a right, duty or unacceptable consequence that the decision must respect.

Connect responsibility to resources

Procurement terms, operational budgets, insurance arrangements and oversight can change who pays for retained competence and remedy. The appropriate arrangement depends on the control an actor has and the duties that apply. This course supplies an analytical map, not jurisdiction-specific legal advice or a claim that all influential actors are fiduciaries.

A funded plan names the resource, the budget holder, the condition for using it and the authority to act. A reserve that requires the original failing team’s consent may not be available when needed. An independent audit without access to evidence may consume its budget without changing a decision. Test the arrangement under the same adverse scenario used to justify it.

Invented for this course

The avoided-cost threshold

In this hypothetical example, an independent reconciliation costs £12,000 a year. A material incident would cost £200,000 to investigate and remedy. The control is assumed to reduce annual incident probability from 10% to 3%, with no other effects.

  1. Expected avoided cost is 0.07 × £200,000 = £14,000 per year under the assumptions.
  2. Subtract the £12,000 control cost: the net expected benefit is £2,000. This is not a measured return or a promise.
  3. The break-even reduction is £12,000 / £200,000 = 6 percentage points. If the reduction is only four points, avoided cost is £8,000 and the net is −£4,000.
  4. Check who pays and who benefits. If the adopter bears the control cost but only one quarter of the avoided incident cost, its private incentive differs from the total calculation.

What follows: The threshold clarifies what evidence matters. It does not establish the probability reduction or settle non-monetary duties.

Attempt before opening feedback

Cost and challenge a safeguard

Create a hypothetical control with an annual cost, an incident consequence and two plausible probability reductions. Calculate the break-even threshold, identify the cost bearers and give one reason the calculation might not decide the issue.

Review criteria

  • All numerical inputs are labelled assumptions unless sourced.
  • Units, time horizon and the break-even calculation are consistent.
  • The analysis distinguishes overall benefit from the adopter’s incentive and includes control costs.
Worked feedback

A strong answer tests a lower-effectiveness scenario and identifies who actually saves money. It may reject the proposed safeguard in favour of a cheaper control that addresses the same failure. It should not claim that a fictional expected value is evidence of a profitable real deployment.

Optional extension · about 15 minutes, additional to core course time

Extension: can the correction plan pay when action is due?

Extend the Chapter 14 funding question by separating an expected annual cost from money available during an incident. A fictional AI-assisted allocation service needs specialists to investigate and correct a failed batch. Every probability, amount, deadline and reimbursement term below is invented for this exercise. This is a planning model, not an insurance quotation or evidence of a real service.

Assume at most one qualifying incident in a year, with a 4% probability. If it occurs, the service must pay £18,000 by day 1 to mobilise the response and another £12,000 by day 10 to complete it. The required staff and inputs are assumed available if paid on time. There is no other incident expenditure in this simplified model. Day 0 is the incident start; all deadlines use the same clock.

The service has £6,000 of uncommitted, immediately usable reserve funds. In this scenario a policy reimburses exactly £24,000 on day 30 after the completed work has been paid for and the claim validated. Assume that reimbursement arrives as specified. It cannot be used early, and no borrowing, supplier credit or other funds are already arranged. Premiums, interest and the opportunity cost of reserved funds are excluded and must be considered before choosing a real arrangement.

Required payments and conditional reimbursement if the incident occurs (invented teaching case)
EventDeadline or receipt datePayment requiredFunds received
Response mobilisationDay 1£18,000£0
Correction completedDay 10£12,000£0
Validated reimbursementDay 30£0£24,000

Attempt first

  1. For this calculation, assume the full response is funded, completed and reimbursed as specified. Calculate the expected annual gross incident expenditure and the expected annual expenditure retained after reimbursement. These are ultimate costs of incidents arising in the year, not necessarily cash paid and received within that calendar year. Which omitted costs prevent either figure from being the total annual cost of this arrangement?
  2. For a year in which the incident occurs, calculate the additional funds needed by day 1 and the peak additional funding needed before day 30. Start with the £6,000 reserve. Do not count the reimbursement before its receipt date. Can the plan as currently funded meet either payment deadline?
  3. A manager proposes reserving only the expected annual gross expenditure, because it is the average cost. Recalculate the two funding gaps. Explain why a valid expectation does not establish that money or response capacity will be available when needed.
  4. Write a four-line funding-and-retest card: name the funding owner, the required amounts and release deadlines, the evidence that funds and response capacity can be used under the failure condition, and a result that would fail the test. Repeat the cash schedule with reimbursement delayed to day 60. Do not assume an unapproved credit line exists.
Funding timeline: calculations and worked feedback

Gross expenditure if the incident occurs is £18,000 + £12,000 = £30,000. Expected annual gross expenditure is 0.04 × £30,000 = £1,200. Retained expenditure after the assumed reimbursement is £30,000 − £24,000 = £6,000 per incident, so its annual expectation is 0.04 × £6,000 = £240. These expectations use the at-most-one-incident assumption and the fully funded, completed and reimbursed response. They concern ultimate costs of incidents arising in the year, not calendar-year cash flows. They exclude premiums, financing costs and the opportunity cost of funds, and do not describe the cash required in an incident year.

By day 1 the initial shortfall is £18,000 − £6,000 = £12,000. By day 10 cumulative expenditure is £30,000, so peak additional funding is £30,000 − £6,000 = £24,000. The day-1 shortfall is part of that £24,000, not an extra amount to add again. With only the stated reserve, the service cannot mobilise on time. Consequently it cannot complete the assumed paid-work-and-validation sequence that would lead to reimbursement. The proposed plan is not executable as funded.

If an authorised source supplies the extra £24,000 on the required schedule, the service can make the assumed payments. Day-30 reimbursement can then repay that temporary funding, leaving £6,000 of net incident expenditure and the original reserve consumed. Arranging such funding has its own cost and conditions. This exercise does not authorise borrowing, spending or purchasing insurance.

If only £1,200 is reserved, the day-1 gap becomes £16,800 and the peak gap becomes £28,800. The 4% probability weights a possible year for the expectation; it does not reduce an actual £18,000 mobilisation bill to 4% of that bill. A provision or budget line also needs an available funding source and someone with authority to release it.

A useful card requires confirmed access to £12,000 of extra funds by day 1 and a further £12,000 by day 10, or a verified alternative that meets both deadlines. Name who can release the funds, any conditions that could block access, and how specialists and necessary evidence will remain available. In an isolated tabletop test, remove the normal approver or a shared service dependency. A required release that arrives late, or a funded specialist who cannot access the records, fails the stated plan. A budget entry alone is insufficient evidence.

Moving reimbursement to day 60 leaves the peak additional principal at £24,000 under the unchanged payment schedule, but extends the financing period by 30 days. Interest, renewal conditions and availability may therefore change; their amounts cannot be calculated from this table. A smaller or refused reimbursement would also change the eventual retained cost. These are separate sensitivities. Funding the plan does not prove the correction works, and a favourable average cannot remove applicable rights, duties or service constraints.

Sources and scope

Source checked: NIST AI RMF 1.0 (2023), MANAGE 2.1, considers resources and viable non-AI alternatives; GOVERN 2.1 concerns clear roles and communication; MANAGE 4.1 includes incident response and recovery in monitoring plans. This original exercise applies those questions to the course’s funding lesson. NIST supplies none of the figures, policy terms, cash-flow calculations or pass criteria here, and does not certify this plan.

Formative practice

Check your reasoning

This is an open self-check, not a secure examination or certificate. Answers and explanations are available below. Nothing you enter is sent to a server.

A £12,000 annual control addresses a £200,000 incident cost. What reduction in annual incident probability breaks even, ignoring other effects?

Explanation and suggested answer

6 percentage points

Divide 12,000 by 200,000 to obtain 0.06, or six percentage points. The calculation depends on the stated consequence and excludes other costs or duties.

Keep your working notes

These notes remain only in this tab. They are not submitted or saved automatically. Download a copy before leaving. Use fictional or public information, not confidential personal or client details.